cipherdyne.org

Michael Rash, Security Researcher



DNS    [Summary View]

NAT and Checking for DNS Cache Poisoning Exploitability

The Dan Kaminsky DNS Checker There are several online DNS checker services that are designed to test nameservers to see if they are vulnerable to the Dan Kaminsky DNS cache poisoning attack. These services perform an important function considering 1) the ease with which vulnerable servers can be exploited, and 2) the critical place DNS has in the Internet infrastructure. If you have not yet checked the nameserver you use for DNS resolution or switched to OpenDNS, let's just say that it is highly recommended, and Dan's talk at the Blackhat Briefings today drove home just how bad the problem is. Dan presented a nice visualization of how much effort has gone into the worldwide patching effort so far, but there is still a lot of work to do.

Online DNS checker services are handy, but unfortunately, not all of these DNS services work the same, and this blog post shows that certain flavors of NAT can make some of these services report nameservers as vulnerable when they really aren't.

Spoiler: The Doxpara and DNS-OARC services handle the NAT case properly by testing against multiple upstream nameservers, whereas the Metasploit test does not yet handle this (although I'm sure it won't be long before this is taken care of).

One of the main requirements for successfully exploiting a vulnerable nameserver is that the source port the server uses for recursive queries must be predictable by the attacker. It's not too difficult for an attacker to profile a potential target by issuing a series of DNS requests for random hostnames (and therefore not cached) in a domain under the attacker's control. Then, by watching how the targeted nameserver issues requests to the authoritative nameserver (controlled by the attacker), it is easy to check whether the source port seems randomly chosen or not. However, if the attacker is only using one authoritative nameserver, this limits the attacker to asking the question:

Are the source ports for recursive queries from the target to a single authoritative nameserver predictable?

...instead of:

Are the source ports for recursive queries from the target across a range of authoritative nameservers predictable?

(For the remainder of this discussion, we'll assume that a targeted nameserver is behind a NAT device, and all recursive DNS requests traverse this device.) The crux of the problem - from the viewpoint of an attacker - is that NAT devices do not always use the same strategy for translating UDP source ports. If the NAT always leaves the source port intact, then there is no problem, and the attacker only needs one authoritative DNS server in order to see whether the target randomizes source ports. If the NAT imposes an "n+m" rule, then this is also advantageous to the attacker with only one authoritative nameserver - even if the targeted DNS server randomizes source ports itself. But, if the NAT randomizes source ports according to a rule like "for the next 30 seconds, give all UDP packets with source port 12345 destined for IP1 a new source port of 48567, and all packets to IP2 a new source port of 34432, etc.", then the attacker with only one authoritative nameserver would see that all incoming queries have the same source port (at least for 30 seconds). Given that an actual DNS cache poisoning attack requires spoofed responses from nameservers that are (presumably) not under the control of the attacker, such a NAT device would thwart an attack because it would assign a new (random) port for the set of queries to each authoritative nameserver.

The iptables firewall with it's SNAT --random feature implements a strategy similar to the above, and we show how a few DNS checker services handle this for a targeted nameserver 22.2.2.2 (the IP is for illustration purposes only).

First, Dan's DNS checker reports the following: Your name server, at 22.2.2.2, appears to be safe, but make sure the ports
listed below aren't following an obvious pattern (:1001, :1002, :1003, or
:30000, :30020, :30100...). Requests seen for c26cd2dfd938.doxdns5.com:
22.2.2.2:40796 TXID=27431
22.2.2.2:28530 TXID=7761
22.2.2.2:56025 TXID=64688
22.2.2.2:30711 TXID=37206
22.2.2.2:24472 TXID=84
Good.

Now, DNS OARC reports: [attacker]# dig @22.2.2.2 +short porttest.dns-oarc.net TXT
porttest.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.pt.dns-oarc.net.
"22.2.2.2 is GREAT: 26 queries in 2.5 seconds from 26 ports with std dev 18972"
This is also good.

Finally, on the exploit side of things, Metasploit's service reports: [attacker]# ./msfconsole
msf > use auxiliary/spoof/dns/bailiwicked_host
msf auxiliary(bailiwicked_host) > set RHOST 22.2.2.2
RHOST => 22.2.2.2
msf auxiliary(bailiwicked_host) > check
[*] Using the Metasploit service to verify exploitability...
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] >> ADDRESS: 22.2.2.2 PORT: 44826
[*] FAIL: This server uses a static source port and is vulnerable to poisoning
msf auxiliary(bailiwicked_host) >
But, the above argument shows that the Metasploit result is a false positive. After seeing such a result, the attacker might continue on with Metasploit and create a ton of DNS queries and spoofed responses, only to have the exploit attempt fail because the source port really isn't predictable for DNS queries issued to nameservers outside the attacker's control.

The prevalence of such NAT deployments is most likely low, but it would make an interesting research project to try and determine how many DNS servers are protected by such a mechanism. However, a key detail is that for testing this is that the DNS servers fundamentally must still select the same source port for recursive queries since if not then the iptables SNAT --random option will be more random (even for rapid sets of recursive queries). Source port randomization is exactly one of the things the patches to DNS servers is designed to address, so the opportunity to perform such research is closing. Also, hopefully no one is still setting query-source address * port within the named.conf file.

Update: 08/09/08 Jeff Jarmoc mentioned to me that the implementations of NAT on Cisco and Check Point devices can de-randomize source ports of services running on internal networks, and this problem even affects DNS servers that have been patched. That is an important point, and the problem was also noted in this blog post. The vendors are patching the issue, but there is a lot of work left.

Metasploit DNS Cache Poisoning and iptables Countermeasures

Metasploit and DNS Cache Poisoning Attacks On July 23rd, H D Moore, I)ruid, and the Metasploit Project released an exploit for the Dan Kaminsky DNS cache poisoning attack - the full details of which will be released at the Blackhat Briefings. We know that source port prediction for recursive queries is a key component to successfully poison a nameserver's cache, and the Metasploit exploit code offers the ability to check a targeted nameserver for predictable source ports. This check is implemented by sending a set of TXT queries against the metasploit.com domain to a targeted server, and - if recursion is enabled - the server will in turn send these queries (which are randomized and therefore not cached) to a nameserver that is authoritative for the metasploit.com domain. The information returned by the metasploit.com nameserver contains the source port the targeted nameserver used to issue the queries. If the source port exhibits a high degree of predictability (such as if the query-source address * port named.conf directive is used or if an external NAT device removes randomness in the source port on its own), then it is much easier to spoof responses to queries against the target from other nameservers and increase the chances that one of these spoofed responses will be cached.

Here is an example of using the source port check feature in Metasploit. Some output has been abbreviated, and the IP addresses 11.1.1.1 (hostname: attacker) and 22.2.2.2 (hostname: target) are used only for illustration purposes: [attacker]# ./msfconsole
msf > use auxiliary/spoof/dns/bailiwicked_host
msf auxiliary(bailiwicked_host) > set RHOST 22.2.2.2
RHOST => 22.2.2.2
msf auxiliary(bailiwicked_host) > check
[*] Using the Metasploit service to verify exploitability...
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] >> ADDRESS: 22.2.2.2 PORT: 30001
[*] FAIL: This server uses a static source port and is vulnerable to poisoning
So, the source port used by the targeted nameserver is always 30001 - convenient.

If we examine the traffic generated by Metasploit on the wire, the TXT queries exhibit some nice structure. Note the spoofprobe-check and .red.metasploit.com strings in both the incoming queries responses (actually, incoming queries contain the bytes |03|red|0a|metasploit|03|com in Snort syntax): [attacker]# tcpdump -i eth0 -l -nn port 53
22:53:40.594082 IP 11.1.1.1.40866 > 22.2.2.2.53: 0+ TXT? spoofprobe-check-1-10913572965.red.metasploit.com. (67)
22:53:40.730809 IP 22.2.2.2.53 > 11.1.1.1.40866: 0 1/1/0 TXT "22.2.2.2:30001 IN IN::TXT spoofprobe-check-1-10913572965.red.metasploit.com" (182)
Further, in the query responses, the string IN::TXT is also returned. So, using this information, we can build iptables rules that leverage the string match extension to inspect application layer data for these strings. We can then have iptables take action such to log or drop packets that match.

If a nameserver is running locally on a Linux system, then the following rules detect inbound requests from the attacker (see the usage of the iptables --hex-string argument to describe the non-printable bytes in the incoming DNS request), as well as responses from the metasploit.com server to outbound recursive requests from the targeted nameserver. Note that you might want to combine the LOG rules with the iptables limit match in order to reduce the number of log messages created during an actual attack. Still, I find that having more data is usually good, and the number of source port reconnaissance queries is much less than the number of spoofed responses when a cache poisoning attempt is made anyway, so it shouldn't be too burdensome to leave off the limit match. [target]# iptables -I INPUT 1 -p udp --dport 53 -m string --string "spoofprobe-check" --algo bm -m string --hex-string "|03|red|0a|metasploit|03|com" --algo bm -j LOG --log-prefix "METASPLOIT DNS RECON QUERY "
[target]# iptables -I INPUT 1 -p udp --dport 53 -m string --string "spoofprobe-check" --algo bm -m string --hex-string "|03|red|0a|metasploit|03|com" --algo bm -j DROP

[target]# iptables -I INPUT 1 -p udp --sport 53 -m string --string "IN::TXT" --algo bm -m string --string "spoofprobe-check" --algo bm -m string --string ".red.metasploit.com" --algo bm -j LOG --log-prefix "METASPLOIT DNS RECON RESP "
[target]# iptables -I INPUT 1 -p udp --sport 53 -m string --string "IN::TXT" --algo bm -m string --string "spoofprobe-check" --algo bm -m string --string ".red.metasploit.com" --algo bm -j DROP
Similar rules can be added to the FORWARD chain (along with specifying the internal subnet or input interface so that directionality can be established) for nameservers that are deployed on a separate system behind a Linux system running iptables. The most important rules above are probably the first two, since matching packets reveal the source IP of the attacker. However, this is of limited use because the actual cache poisoning attack will involve packets spoofed from other authoritative nameservers, and it is possible to collect source port information from other sources. Still, having information about someone doing source port predictability reconnaissance against one of your nameservers with Metasploit is worth knowing.

With the rules above in place, the 'check' step in Metasploit is unable to tell that the targeted nameserver even responds to recursive queries at all, and back on the firewall system several METASPLOIT DNS RECON QUERY log messages are written to syslog by iptables: msf auxiliary(bailiwicked_host) > check
[*] Using the Metasploit service to verify exploitability...
[*] ERROR: This server is not replying to recursive requests

[target]# tail /var/log/messages
Aug 2 06:39:55 target kernel: [933142.545502] METASPLOIT DNS RECON QUERY IN=eth0 OUT= MAC=00:13:46:3b:41:4c:00:12:46:c2:60:44:09:00 SRC=11.1.1.1 DST=22.2.2.2 LEN=96 TOS=0x00 PREC=0x00 TTL=63 ID=33573 DF PROTO=UDP SPT=40273 DPT=53 LEN=76
Aug 2 06:39:55 target kernel: [933142.637446] METASPLOIT DNS RECON QUERY IN=eth0 OUT= MAC=00:13:46:3b:41:4c:00:12:46:c2:60:44:09:00 SRC=11.1.1.1 DST=22.2.2.2 LEN=96 TOS=0x00 PREC=0x00 TTL=63 ID=33574 DF PROTO=UDP SPT=40273 DPT=53 LEN=76
On another note, before using Metasploit to test your DNS infrastructure to see if it is exploitable, it is important to know whether your local network allows spoofed packets out. Many firewalls can be configured to drop spoofed packets from internal systems, and even my little LinkSys router does this. Because there is no mechanism in Metasploit currently (as far as I know) to detect whether your local network filters spoofed packets (and building such a mechanism would be tricky for various technical reasons), an incorrect assumption can result in a cache poisoning attack that has no possibility of succeeding but that also generates thousands of DNS queries at the same time.

Finally, for those unpatched nameservers running behind an iptables firewall, the SNAT --random option can provide a work-around for predictable source ports. Nevertheless, the emphasis should always be on patching vulnerable servers since the source port problem is only one aspect of the vulnerability in DNS.

Mitigating DNS Cache Poisoning Attacks with iptables

Digg Mitigating the Kaminksy DNS attack Kaminsky DNS Exploit It is well known that last week Dan Kaminsky publicized a cache poisoning exploit against DNS. The details of this exploit have not yet been released, but Dan will present a talk at the Blackhat Briefings next month that will clarify the technical specifics behind the vulnerability - and therefore how to exploit it. Even though Dan has yet to release the details, enough information has already been released to show that iptables may be able to mitigate the vulnerability with a single well-chosen rule in your iptables policy. Further, this technique can be used right now whenever a vulnerable DNS server is deployed on (or behind) a system running iptables. Although updates have been released in a coordinated effort for most platforms, not everyone patches their systems immediately, and in some cases it can be easier to execute a few iptables commands than to deploy such updates. Of course, patching the vulnerability should be the top priority, but the exact details of the attack haven't been disclosed so it is hard to gauge risk. That said, if Dan Kaminsky says to patch, then it's probably a good idea to do so. In the meantime, let's press on.

The CERT advisory highlights three problems in the existing DNS infrastructure:

  1. Lack of sufficient randomness in the selection of source ports for DNS queries
  2. DNS transaction ID values that also exhibit insufficient randomness
  3. Multiple outstanding requests for the same resource record
For this blog post we are concerned about source port predictability for DNS queries, but Dan's cache poisoning attack (most likely) relies heavily on the first two problems in combination. Dan has probably found a clever way to weaponize his attack so that it can be leveraged against lots of DNS servers.

How can iptables help? First, let's assume that source port predictability is a necessary prerequisite to poison a DNS cache by Dan's technique. So, if iptables can introduce additional randomness into the source port that bind uses - at least as far as any upstream DNS server can tell - for each DNS query then an attacker would not be able to rely on source port predictability. The iptables SNAT target supports the ability to randomize both TCP and UDP source ports (see the --random option) even if a userspace application chooses a specific source port. This applies to both locally generated and forwarded packets. Hence, if a piece of userland software (such as bind) uses predictable source ports, iptables can rectify this via the SNAT target subject to some connection tracking restrictions.

To illustrate this, we use nmap with its --source-port option to set the source port to 44444 for a UDP scan of port 53 on two different servers, and verify with a packet trace that the source port is indeed set to this value. We'll assume the IP's 22.2.2.2 and 33.3.3.3 simulate upstream DNS servers from a local DNS server on the system with IP 11.1.1.1: # nmap --source-port 44444 -P0 -p 53 -sU 22.2.2.2 33.3.3.3

19:26:19.625637 IP 11.1.1.1.44444 > 22.2.2.2.53: [|domain]
19:26:19.625790 IP 11.1.1.1.44444 > 33.3.3.3.53: [|domain]
19:26:19.729520 IP 11.1.1.1.44444 > 33.3.3.3.53: [|domain]
19:26:20.626527 IP 11.1.1.1.44444 > 22.2.2.2.53: [|domain]
As you can see, the source port for each UDP datagram from 11.1.1.1 is set to 44444. Now, to force the source port to be randomized when each packet is transmitted despite setting it to 44444 from the nmap command line, let's use the SNAT target with the --random option (note that it is assumed that the iptables policy is also using connection tracking to allow expected replies through as well, etc.): # iptables -t nat -I POSTROUTING 1 -p udp -s 11.1.1.1 --dport 53 -j SNAT --to 11.1.1.1 --random By executing the same nmap command again, now we see that the SNAT target has changed the source port on the outgoing UDP datagrams to 9374 (for 22.2.2.2) and 54743 (for 33.3.3.3): # nmap --source-port 44444 -P0 -p 53 -sU 22.2.2.2 33.3.3.3

19:28:05.613637 IP 11.1.1.1.9374 > 22.2.2.2.53: [|domain]
19:28:05.613792 IP 11.1.1.1.54743 > 33.3.3.3.53: [|domain]
19:28:05.717536 IP 11.1.1.1.54743 > 33.3.3.3.53: [|domain]
19:28:06.617553 IP 11.1.1.1.9374 > 22.2.2.2.53: [|domain]
Hence, iptables can introduce randomness into source ports via a NAT operation regardless of whether a userspace application requests a specific source port.

However, you may have noticed that nmap generates two packets for each scan of UDP/53, and you may have also noticed that the source port is 9374 for both packets sent to 22.2.2.2, and 54743 for both packets sent to 33.3.3.3. While the original source port of 44444 has definitely been randomized, it appears that this randomization is not taking place on a per-packet basis. What is really happening is that iptables has to use its connection tracking mechanism to map packets translated with the SNAT target to the correct socket, and this mapping is created from the srcIP/dstIP/sport/dport/protocol tuple along with a set of timers (see the nf_ct_udp_timeout and nf_ct_udp_timeout_stream variables in the kernel linux/net/netfilter/nf_conntrack_proto_udp.c file). So, if multiple requests are made to the same upstream DNS server within a 30 second time window, then the SNAT --random rule will map the source port for each request to the same (randomly assigned) source port - the initial tuples are the same after all.

At first glance this might seem to be a show stopper, but assuming that Dan's cache poisoning attack requires a process of querying different DNS servers - at least one of which is under the control of the attacker so that the source port behavior of the targeted server can be monitored - then the SNAT --random strategy would provide an effective defense. That is, the iptables state tracking code will assign a different random source port on (at least) a per-server basis even for rapid sets of DNS queries. Further, for queries to the same DNS server that are 30 seconds apart, the source port will change on a per-query basis as well.

This should be an effective defense against the attack, but we'll have to wait for Blackhat to be sure.

On another note, applications commonly do not bother requesting specific source ports for client sockets because they assume that the networking stack provided by the local kernel will assign a sensible source port. Or, they don't have to worry about the source port because they build in enough security at the application layer (unlike DNS implementations that don't sufficiently randomize transaction ID's) such that source port prediction does not give an advantage to the attacker. Still, those stacks that don't choose a random source port are not helping the state of network security, and the Kaminsky DNS attack is a perfect example of why. It turns out that the Linux kernel has only recently started randomizing UDP source ports as of 2.6.24.

Update 07/16/2008: Jon Hart has also written a blog post that illustrates using OpenBSD's pf firewall to implement a similar mitigation strategy. Hence, multiple pieces of firewall infrastructure can offer effective protection from the cache poisoning attack.