DNS records are incorrect or missing, causing clients to resolve the wrong IP address, fail to reach a service, or receive stale cached answers after a record has been updated.

Common triggers:

A record was added, changed, or deleted at the DNS provider but has not yet propagated globally.
A cached record at a public resolver (Cloudflare 1.1.1.1, Google 8.8.8.8) is serving a stale answer.
The record was never created, or was created under the wrong name/type/zone.

Impact

End-users or services cannot connect to the affected hostname.
Traffic may be routed to a decommissioned or incorrect endpoint.
TLS certificate issuance (ACME/Let's Encrypt) may fail if DNS-01 or HTTP-01 challenges cannot resolve.
Incident duration extends by the TTL of the stale record at each resolver that has cached it.

Diagnosis

1. Confirm the expected record

Check the DNS provider's control plane (e.g. Azure DNS, Cloudflare) and verify:

The record exists under the correct zone.
The type (A, CNAME, TXT, MX, …) and value are correct.
The TTL is set appropriately (low TTL, e.g. 60–300 s, during active changes).

2. Check propagation globally

Use whatsmydns.net to query the record from resolvers across multiple regions simultaneously.

Enter the hostname and select the record type.
A mix of old and new values indicates propagation is still in progress.
Unanimous stale/wrong answers indicate the record at the source has not been fixed yet, or public resolvers have not been flushed.

3. Query authoritatively

Run a direct query against the zone's authoritative nameserver to bypass any caching layer:

# Find the authoritative nameserver for the zone
dig NS <zone> +short

# Query it directly
dig @<authoritative-ns> <hostname> <type>

If the authoritative server returns the wrong answer, the record itself needs to be corrected (go to Mitigation → Fix the record). If the authoritative server returns the correct answer but public resolvers do not, proceed to flush the cache.

Mitigation

Fix the record

Log in to the DNS provider's control plane.
Navigate to the correct zone.
Create, update, or delete the record as required.
Set a short TTL (60–300 s) while changes are in flux; restore to the standard value (e.g. 3600 s) once stable.
Confirm the authoritative nameserver now returns the correct answer:
```
dig @<authoritative-ns> <hostname> <type>
```

Important

DNS records are managed via automation for Gen3 infrastructure: special care needs to be taken to adopt the recovered record back to external-dns

Flushing the public DNS cache

After the authoritative record is correct, flush the two major public resolvers so they stop serving stale answers.

Cloudflare (1.1.1.1)

Go to https://one.one.one.one/purge-cache/, enter the hostname and record type, and click Purge Cache.

Google Public DNS (8.8.8.8)

Go to https://dns.google/cache, enter the hostname, and click Flush Cache.

Note

Both resolvers must be flushed — clients may use either one depending on their OS or network configuration.

System and network caches

Even after public resolvers are flushed, clients may still resolve stale answers because of caching layers that the flush tools cannot reach:

OS DNS cache — Windows, macOS, and Linux (systemd-resolved) each maintain a local cache. Flush it on the affected machine:
- macOS: sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder
- Linux (systemd-resolved): sudo resolvectl flush-caches
- Windows: ipconfig /flushdns
Network / router cache — corporate gateways, VPN concentrators, and ISP-level resolvers cache records independently and cannot be purged remotely. Clients behind these layers must wait for the TTL to expire.
Application-level cache — some services (e.g. Java runtimes, Node.js) cache DNS results in-process. A process restart is required to pick up the new record.

Note

Flushing the two public resolvers (Cloudflare, Google) covers globally distributed traffic, but a reporter who still sees the old IP is most likely hitting one of the above local caches, not a propagation problem.

Verify propagation

Re-check whatsmydns.net after flushing. Most resolvers should now return the correct value within a few minutes.
Confirm end-to-end connectivity from an affected client:
```
curl -v https://<hostname>/
```
If some resolvers still show stale data, wait for their TTL to expire — flush tools clear the resolver's own cache but cannot force every downstream caching layer to refresh immediately.