Services

Resources

Company

Debugging a website that worked on every device except mine

Debugging a website that worked on every device except mine.

Storytime!

Colleague: Hey, can anyone help? I can't access go.dev on my work laptop. Tried different browsers, cleared DNS cache, nothing works.

You: When you say unable to access - do you mean you're getting an HTTP error or DNS is not resolving?

Colleague: Browser says it can't resolve the hostname. Even tried Safari - same issue.

You: Was it working before? What changed recently?

Colleague: Yes, it worked before. I switched from OpenVPN to Tunnelblick recently, can't think of anything else.

You: Can you try docker run -it ubuntu bash and check go.dev from there?

Colleague: Doesn't work! Even inside Docker.

You: Let's get on a call.

On the call...

You: Run scutil --dns and see what we get.

Colleague: There are entries with domains like <company-name-service>.dev. That's weird.

You: Try curl go.dev.

Colleague: "Could not resolve hostname" error.

You: But dig go.dev?

Colleague: That returns correct DNS records.

You: So something on your local machine is intercepting DNS queries. The Docker failure confirms this - containers inherit host DNS config.

Colleague: Wait, I installed KubeVPN a few days ago using brew install kubevpn. It let me access Kubernetes services directly instead of port-forwarding.

You: Ah! KubeVPN hijacks DNS resolution. It routes .cluster.local domains to your Kubernetes cluster's DNS server.

Colleague: Oh no. We have a namespace called dev in our cluster.

You: Exactly! So when you try go.dev, the system looks for:

  • go.dev.dev.svc.cluster.local

  • go.dev.svc.cluster.local

  • go.dev

Since there's no "go" service, DNS fails completely.

Colleague: Should I run kubevpn disconnect?

You: Not sure whether it will clean up your local records. On macOS, there are system-wide DNS resolvers AND per-network-adapter resolvers. KubeVPN probably modified the per-adapter settings.

Let's reset DNS for each network interface:

services=$(networksetup -listallnetworkservices | grep 'Wi-Fi\\|Ethernet\\|USB')
while read -r service; do
    networksetup -setdnsservers "$service" 1.1.1.1 1.0.0.1
done <<< "$services"
services=$(networksetup -listallnetworkservices | grep 'Wi-Fi\\|Ethernet\\|USB')
while read -r service; do
    networksetup -setdnsservers "$service" 1.1.1.1 1.0.0.1
done <<< "$services"
services=$(networksetup -listallnetworkservices | grep 'Wi-Fi\\|Ethernet\\|USB')
while read -r service; do
    networksetup -setdnsservers "$service" 1.1.1.1 1.0.0.1
done <<< "$services"
services=$(networksetup -listallnetworkservices | grep 'Wi-Fi\\|Ethernet\\|USB')
while read -r service; do
    networksetup -setdnsservers "$service" 1.1.1.1 1.0.0.1
done <<< "$services"
services=$(networksetup -listallnetworkservices | grep 'Wi-Fi\\|Ethernet\\|USB')
while read -r service; do
    networksetup -setdnsservers "$service" 1.1.1.1 1.0.0.1
done <<< "$services"

Colleague: Running it... Wow! This fixes the problem, go.dev loads immediately!

You: There you go. This sets all network services to use Cloudflare's public DNS.

Colleague: So updating /etc/resolv.conf wouldn't have fixed this?

You: Correct. You have to fix per-adapter settings through networksetup.

Colleague: This is embarrassing. I didn't understand what KubeVPN was doing.

You: Key takeaways:

  • DNS on macOS has multiple layers.

  • Tools that seem like magic usually ARE doing complex things behind the scenes.

  • Don't run commands without understanding what they do.

Colleague: And container networking isn't always isolated.

You: Right. Most failures have DNS as a root cause. Check DNS configuration first!

I write such stories on software engineering.

There's no specific frequency, as I don't make these up.

If you liked this one, you might love - A curious case of Postgres choosing the wrong index

Follow me on LinkedIn and Twitter for more such stuff, straight from the production oven!

Subscribe for more such content

Get the latest in software engineering best practices straight to your inbox. Subscribe now!

Hold for 2 seconds to verify

Security powered by One2N

Subscribe for more such content

Get the latest in software engineering best practices straight to your inbox. Subscribe now!

Hold for 2 seconds to verify

Security powered by One2N

Subscribe for more such content

Get the latest in software engineering best practices straight to your inbox. Subscribe now!

Hold for 2 seconds to verify

Security powered by One2N

Continue reading.