Kubernetes DNS Security

Key Takeaways:

• A publicly resolvable wildcard A record for apps.example.com reveals every internal hostname to the internet, even when the targets cannot be reached externally. Removing it eliminates that exposure without affecting access.
• The pattern requires three components: a dedicated CoreDNS deployment managing one internal zone, the Tailscale Kubernetes Operator exposing only that resolver to the tailnet, and a split DNS entry limited to a search domain.
• Do not reuse the cluster’s kube-dns resolver. Exposing it to the tailnet makes internal service domains such as svc.cluster.local visible to all devices on the network.
• Verification involves two checks: dig from a device on the tailnet returns the record, and the same query from outside the tailnet returns NXDOMAIN.
• DNS rebinding protection, CoreDNS hairpin routing, and short TTLs are the three failure modes that can disrupt this setup in production.

Why a Public Wildcard Record Is the Wrong Default

A wildcard A record for apps.example.com is a common Kubernetes platform shortcut. One record covers every service, and internal users get memorable hostnames without per-service DNS changes. The drawback is that the record is public. Anyone can query it, enumerate the names beneath it, and map the structure of your internal platform.

This exposure occurs even when the addresses are not reachable from outside. A wildcard resolving to a private address reveals the naming scheme, service inventory, and environment layout. Penetration test reports and cloud security posture reviews often identify this: an internal namespace published to public resolvers provides information useful for lateral movement planning.

Split DNS is the standard solution. It directs queries for a specific domain suffix to a resolver you control while leaving public resolution unchanged for other domains. Tailscale’s documentation explains the mechanism clearly: you send queries for every domain ending in an internal suffix to an internal server “so that you can privately expose services without DNS names being visible to public internet,” a practice that also helps prevent DNS cache poisoning attacks against internal services (Tailscale, What is Split DNS).

The following walkthrough uses Amazon EKS as an example. The same three components (dedicated resolver, operator exposure, split DNS rule) apply to any Kubernetes distribution, including self-managed clusters, EKS Anywhere, and managed offerings from other clouds. EKS is chosen because the CoreDNS add-on and operator installation paths are well documented.

Tailscale Mesh VPN on Amazon EKS, in Plain Terms

Tailscale is a mesh, zero-trust overlay network built on WireGuard, and the distinction from a conventional VPN matters here. A traditional hub-and-spoke setup tunnels all traffic through a central gateway. Tailscale builds peer-to-peer encrypted links between devices, using a coordination service to exchange keys and network positions while traffic flows directly. AWS’s writeup of the EKS Hybrid Nodes integration states the difference directly: “Unlike traditional VPNs, which tunnel all network traffic through a central gateway server, Tailscale creates a peer-to-peer mesh network… It enables encrypted point-to-point connections using open source WireGuard protocol” (AWS Containers Blog).

Every device on the network, called a tailnet, receives a stable address from the 100.64.0.0/10 carrier-grade NAT range, which Tailscale documents as a private range not used on the public internet (Tailscale IP pool docs). Identity comes from your existing SSO provider, and access is expressed as policy over users, groups, devices, and tags rather than IP allowlists. A device joins because an identity authenticated, not because it sits on a trusted subnet.

The trade-offs deserve equal billing. The control plane is hosted by Tailscale, so devices depend on a third party to discover each other, even though the data path is peer-to-peer. An independent 2026 review names this as the product’s most significant architectural limitation: while data traffic is peer-to-peer and encrypted, “the ‘control plane’… is managed by Tailscale” (AI Indigo review). When direct peer-to-peer connections cannot be established, traffic falls back to Tailscale’s DERP relay servers, which keeps the session encrypted but can reduce throughput; the same review notes restrictive firewalls and carrier-grade NAT can force that fallback. For a DNS resolver handling query traffic, relay fallback is acceptable. For bulk data transfer through the same overlay, measure before you commit.

Raw WireGuard is the alternative. It is the same protocol, it is free, and it gives you complete control of the key exchange. What you give up is the coordination layer: peer discovery, NAT traversal, key rotation, and identity-based policy all become your problem. For a handful of static peers, hand-managed WireGuard is reasonable. For a fleet of laptops, CI runners, and ephemeral pods that need to find each other across changing networks, the coordination service is the product.

Architecture: A Private Resolver Replacing a Wildcard Record

The design has three moving parts, and the order matters because each depends on the previous one.

First, a dedicated CoreDNS deployment that owns one internal zone. This resolver serves authoritative records for internal.example.com only. It is separate from the cluster’s kube-dns resolver, and that separation is not optional. The kube-dns resolver answers for cluster-internal domains such as svc.cluster.local and cluster.local. Exposing it to the tailnet would publish those internal service domains to every device on the network.

Second, the Tailscale Kubernetes Operator exposes only that resolver. A Kubernetes Service carrying the correct Tailscale annotations and load balancer class tells the operator to create an ingress proxy pod. The operator assigns that pod a 100.x tailnet address, and the resolver becomes reachable at that address from anywhere on the tailnet.

Third, split DNS in the admin console forwards the internal zone to that address. The rule is scoped to a search domain, so only queries matching internal.example.com go to the private resolver. Everything else resolves normally.

The net effect is a clean split. A device on the tailnet resolves app.internal.example.com and reaches the service. A device outside the tailnet asks the same question of public DNS and receives NXDOMAIN, because no public record exists. The wildcard record for apps.example.com can then be deleted, which closes the exposure finding and removes the need for a public load balancer to front internal-only services.

Build the Dedicated CoreDNS Zone

CoreDNS is configured through a Corefile, a text file defining which zones the server is authoritative for and which plugins handle each. The file plugin serves zone data from an RFC 1035-style master file, which suits a small, hand-managed internal zone (CoreDNS file plugin docs).

A minimal Corefile declares authority over one zone, serves records from a zone file, caches answers, and logs queries for audit.

internal.example.com {
 file /zones/internal.example.com.zone
 cache 30
 log
 errors
}

The matching zone file holds the records for each ingress hostname you want resolvable. A short TTL is the right default, because the point of the private resolver is that you can change a record and have it propagate quickly without waiting on public DNS caching.

Note: The following code is an illustrative example and has not been verified against official documentation. Please refer to the official docs for production-ready code.

$ORIGIN internal.example.com.
$TTL 30
@ IN SOA ns1.internal.example.com. hostmaster.internal.example.com. (
 2026091801 ; serial
 3600 ; refresh
 600 ; retry
 604800 ; expire
 30 ) ; minimum
@ IN NS ns1.internal.example.com.
ns1 IN A 100.x.x.x
app IN A 100.x.x.x
api IN A 100.x.x.x

Note what this example does not handle: no DNSSEC signing, no zone transfer, no automated record updates when a new ingress is created. For a deployment that changes frequently, either generate the zone file from your ingress inventory in a GitOps pipeline, or move to the ExternalDNS pattern described below. The static zone file is the starting point, not the end state.

Deploy this as a Deployment with a ConfigMap holding the Corefile and a second ConfigMap holding the zone file, mounted read-only into the container. Two replicas is a reasonable floor for a resolver that internal tooling depends on. Point the Service at port 53 on both TCP and UDP; DNS falls back to TCP for large responses and for some client resolvers.

Expose Only That Resolver to the Tailnet

The Tailscale Kubernetes Operator turns a cluster Service into a tailnet-reachable endpoint. Install it with an OAuth client scoped to tag:k8s-operator, and define both tag:k8s-operator and tag:k8s under tagOwners in your access control policy so the operator can manage the devices it creates (Tailscale operator install docs).

To expose the resolver, annotate its Service and set the load balancer class. The operator watches for Services marked this way and creates an ingress proxy pod, which uses iptables or nftables rules to forward traffic bound for the proxy’s tailnet address to the Service’s internal ClusterIP (operator architecture docs).

apiVersion: v1
kind: Service
metadata:
 name: internal-dns
 namespace: dns
 annotations:
 tailscale.com/hostname: internal-dns
 tailscale.com/proxy-class: dns-resolver
spec:
 selector:
 app: internal-dns
 ports:
 - name: dns-udp
 port: 53
 protocol: UDP
 targetPort: 53
 - name: dns-tcp
 port: 53
 protocol: TCP
 targetPort: 53
 type: LoadBalancer
 loadBalancerClass: tailscale

The tailscale.com/proxy-class annotation links the Service to a ProxyClass, a cluster-scoped resource that customizes the proxy pods the operator creates. Use it to pin resource requests, apply labels, and set the security context on the resolver proxy (ProxyClass docs). Without it, the proxy inherits defaults that may not suit a DNS workload.

apiVersion: tailscale.com/v1alpha1
kind: ProxyClass
metadata:
 name: dns-resolver
spec:
 statefulSet:
 pod:
 labels:
 app.kubernetes.io/component: tailscale-dns-proxy
 tailscaleContainer:
 resources:
 requests:
 cpu: 100m
 memory: 128Mi

After applying both, check the Service for its assigned address. The EXTERNAL-IP field populates with a 100.x tailnet address once the proxy pod joins the network. That address is the one you enter in split DNS, and it is the only part of the cluster exposed to the tailnet for this purpose.

Split DNS in the Admin Console

Split DNS in Tailscale is configured as a restricted nameserver: a nameserver that applies only to queries matching a specific search domain. The admin console steps are short.

• Open the DNS page of the Tailscale admin console.
• Select Add nameserver, then Custom.
• Enter the 100.x address assigned to the resolver Service.
• Enable Restrict to search domain.
• Enter internal.example.com as the domain.
• Save.

Tailscale’s documentation describes the resulting behavior with a concrete example: configuring a nameserver for example.com tells devices to use that server “only to look up DNS queries that match *.example.com” (DNS in Tailscale). Search domains require Tailscale client version 1.34 or later, so confirm your fleet meets this requirement before relying on short-name resolution.

Two related settings interact with this. The Override DNS servers toggle forces devices to use tailnet DNS settings instead of their local ones, which guarantees internal names resolve consistently. It also means a device that cannot reach your global nameservers will fail to resolve anything, so verify ACLs permit DNS traffic before enabling it. Separately, if any devices use an exit node, the exit node becomes their resolver for all domains by default unless you explicitly mark the nameserver for use with exit nodes.

Verification: On-Tailnet Records, Off-Tailnet NXDOMAIN

Verification is a two-sided test, and both sides must pass before you delete the public wildcard. Run the first query from a device connected to the tailnet.

Note: The following code is an illustrative example and has not been verified against official documentation. Please refer to the official docs for production-ready code.

dig app.internal.example.com +short
# expected: 100.x.x.x

dig app.internal.example.com
# check the TTL in the ANSWER SECTION; expect 30

The record should return the tailnet address with the low TTL you configured. A low TTL is deliberate: it lets you change a record and have clients pick up the change within seconds rather than hours, which matters when rotating an ingress address during an incident. Now run the same query from a device that is not on the tailnet, ideally from a network with no route to your cluster.

Note: The following code is an illustrative example and has not been verified against official documentation. Please refer to the official docs for production-ready code.

dig app.internal.example.com @1.1.1.1
# expected: status: NXDOMAIN

NXDOMAIN is the correct answer and the security property you are buying. The name does not exist in public DNS, so there is nothing to enumerate and nothing to poison. Once both tests pass, delete the wildcard record for apps.example.com. Confirm the deletion with the same off-tailnet query against the wildcard itself, then re-run your internal access checks to make sure nothing depended on the public path.

Use dig rather than nslookup for this work. Tailscale’s documentation warns that on some platforms nslookup bypasses the operating system’s DNS configuration and returns incorrect results, which is exactly the situation you hit when testing split DNS (DNS in Tailscale).

Gotchas: Rebinding, Hairpin NAT, TTL, ACLs

DNS rebinding protection. Some DNS servers and home routers block responses containing private IP addresses to prevent certain attacks. Tailscale’s documentation notes that some servers apply this policy to the Tailscale address range as well, since it falls under RFC 6598 (Tailscale DNS rebinding FAQ). The symptom is a device that cannot resolve internal names from one network but works fine from another. The safest fix is the split DNS configuration itself, which routes those queries to a resolver inside the tailnet and avoids the external server. Disabling rebinding protection entirely is not recommended; excluding a single domain is a better approach.

Hairpin and NAT routing. If your resolver returns a private address reachable only through a subnet router, clients must have accepted the advertised routes, and the router must reach the resolver. Tailscale’s split DNS guidance covers the NAT hairpinning case directly: a service behind a NAT gateway that cannot be reached by its public address needs a separate DNS answer for internal clients (What is Split DNS). Test resolution and reachability separately, because a correct DNS answer to an unreachable address looks identical to a broken resolver until you try to connect.

TTL choice. A 30-second TTL keeps changes fast but increases query volume. A 300-second TTL reduces load and slows rollback. Pick based on how often your ingress addresses change; if they change only during deployments, a moderate TTL is fine. Do not set a long TTL on records you might need to pull during an incident.

ACL and tag setup. The operator tags the devices it creates, and by default uses tag:k8s for both proxy devices and the Tailscale Services they advertise (operator RBAC docs). Because devices and services draw from the same tag namespace, a grant targeting that shared tag applies to both. Write your access policy so only the client groups that need internal resolution can reach the resolver on port 53, and keep the operator’s own tag separate from the tag applied to the resolver proxy.

Auth key and device expiry. Auth keys expire by default, and a device whose key has lapsed silently drops off the tailnet. A practitioner writeup of Tailscale on AWS describes the failure mode exactly: an instance launches, the application runs, and the device never appears in the admin console because the key expired months earlier (yaw.sh, Tailscale on AWS gotchas). For the operator, use OAuth clients rather than long-lived auth keys so there is no manual rotation to forget.

When This Pattern Wins, and When It Does Not

Three approaches solve overlapping problems, and the choice depends on how many services you have and whether you need TLS on the internal names.

Tailscale documents the third approach in detail: a Gateway API implementation with ExternalDNS managing records in an internal resolver such as CoreDNS or Pi-hole, and cert-manager provisioning certificates for custom domains (Tailscale BYOD Gateway guide). That pattern is heavier: it adds a gateway, a certificate issuer, and an ExternalDNS deployment. It pays off when you have dozens of services and want records created automatically as part of a deployment.

The whole-zone resolver fits when the constraint is naming rather than per-service policy. If your teams already expect app.internal.example.com to work, and you want one component to own that namespace, a single dedicated resolver is less machinery than a gateway plus ExternalDNS plus cert-manager. It also keeps the change surface small: one Service, one ProxyClass, one admin console entry. The cost is that you own the zone file, and if it drifts from reality you get stale records.

Per-service ingress fits when the number of services is small and you want access policy granularity per service. Its limitation is that every service gets a ts.net hostname unless you add your own domain layer on top, and each exposed service is another proxy pod to run and monitor.

Troubleshooting Checklist

When internal names stop resolving, work through these in order. The first three catch the majority of incidents.

• Is the device on the tailnet? Check tailscale status. If the device is stopped or its key expired, nothing downstream will work.
• Does the resolver Service have a tailnet address? Check the EXTERNAL-IP field. If it is empty, the proxy pod has not joined the tailnet; check the operator logs and the OAuth client scope.
• Does the split DNS rule match the query? Confirm the search domain in the admin console is the exact zone you are querying, not a parent domain that routes more traffic than you intended.
• Is the query reaching the resolver? Query the resolver address directly with dig @100.x.x.x app.internal.example.com. If this works but the plain query does not, the problem is in the split DNS rule or client configuration, not the resolver.
• Is rebinding protection interfering? Test from a different network. If the failure follows the network rather than the device, rebinding protection is the likely cause.
• Is a route needed? If the resolver returns an address reachable only through a subnet router, confirm the router’s routes are approved and that clients have accepted them.
• Is the client caching a failure? Flush the local DNS cache before concluding the server side is broken.

For more on overlay network behavior and the failure modes that show up when peer-to-peer paths degrade, see our earlier coverage of Tailscale peer relay troubleshooting in production and the peer relay deployment guide. The access-control model that underpins this design is the same one described in our analysis of zero trust network access, where enforcement moves from the network boundary to per-identity policy.

The pattern is one way to solve internal name resolution, and it fits when the requirement is a private namespace, a small number of components, and a clear definition of who can resolve your internal hostnames. Delete the public wildcard last, and only after both verification queries return the expected results.

Related Reading

More in-depth coverage from this blog on closely related topics:

• Implementing Kruskal’s Algorithm in Go

Sources and References

Sources cited while researching and writing this article:

• What is Split DNS & Why Should You Use It? – Tailscale
• Simplify network connectivity using Tailscale with Amazon EKS Hybrid Nodes | Containers
• Tailscale IP pool docs
• AI Indigo review
• file – CoreDNS
• Install the Tailscale Kubernetes Operator · Tailscale Docs
• Tailscale Kubernetes Operator architecture · Tailscale Docs
• Use ProxyClass for proxy customization · Tailscale Docs
• DNS in Tailscale · Tailscale Docs
• DNS problems with internal services and DNS rebinding protection
• Permissions and RBAC for the Tailscale Kubernetes Operator · Tailscale Docs
• Tailscale on AWS: The Gotchas Nobody Warns You About – yaw
• Use custom domains with Kubernetes Gateway API and Tailscale