Too Far Afield

I just got an email from Feargal Mac Conuladh, Vice President, Lenovo Services EMEA, who asked for feedback regarding Lenovo’s service. The service was powered by IBM and the quality was good overall. A few things would have improved my experience, though:

• The German phone number of IBM was buried deep inside an English Lenovo web site and I spent some time tracking it down.
• I had to reprogram the telephone system so that I was able to call the expensive service line of IBM. Also, I was on hold for 5 minutes. I don’t know if I’m that comfortable with paying IBM for something that is somehow their fault (waiting for a free agent, hardware problem).
• The call centre agent at IBM said, a technician would call me to make an appointment. He didn’t. Instead he just came and it was a coincidence that someone was on site to open the door. And that my notebook was there, too. I wasn’t at the time.

On the positive side, there were no weird questions from the call centre agent trying to diagnose, if my problem was really a problem. She just asked in which city I was at the moment and that was it, basically. Also, the technician replaced the faulty fan in no time. The response time was not very quick, but not bad either. It was okay per the terms of the SLA. Fast, competent, hassle-free customer service. Thumbs up, IBM!

Yesterday afternoon at 4 p.m. I finally phoned IBM regarding the broken fan in my ThinkPad. About an hour ago, a technician showed up and replaced the broken fan. Now I’ve got a silent office again.

Right now our DNS servers are used for a DNS root query amplification attack. The attack works like this: Mallory (the bad guy) is sending a lot of UDP queries for the root zone to a number of DNS servers. These queries are very short and don’t require a lot of bandwidth. In theory, the DNS server will respond with a very large response, because the response would contain the whole DNS root zone file. When sending the queries, Mallory will spoof the IP address of the source. He uses the IP of Alice (the good gal). The DNS server will send all the large root zone files back to Alice. Alice connection will be filled with root zone files from all over the world. This is a very nice attack vector, as it allows Mallory to send much more traffic to Alice then he can actually generate himself. Also, this does not really affect the DNS servers that are used by Mallory. If he does it the right way, it might even go unnoticed.

According to our DNS server log, these forged queries come in at a rate of about 20 Hz. Of course, our DNS server does not allow recursion from outside the trusted company network, so this is not really a problem either for us nor for Alice. Our DNS servers respond with a servfail. The log is full of these errors at the moment:

1. Not authoritative for '', sending servfail to 89.149.221.182 (recursion was desired)

It is important to note that the IP address displayed in the log (which is in the middle of a subnet of a German provider based in Frankfurt/Main) is not Mallory, the attacker. It is Alice, the target of the attack.

While not as bad as a full root zone response, a servfail is still data. If we send enough data down the line to Alice, their service will go down. Also, our own log files will be full of error messages and they will grow big. We don’t want any of this, so what else can we do beyond disabling recursion for the outside world? Of course we could stop responding to those kind of requests entirely, playing stealth DNS. Unfortunately, this makes DNS cache poisoning easier, so we don’t want to do this. Instead, we use fail2ban to block an IP address temporarily after 10 failed queries for the root zone in a relatively short time frame. The following regular expressions will be fine for PowerDNS:

1. not authoritative for(.*)servfail(.*)<HOST>
2. pdns\[.*\]: Not authoritative for '', sending servfail to <HOST>

If you are administering your own DNS server, check your log files for these kind of attacks and implement appropriate measures to prevent them.

Recently, I ate in a restaurant car of ?eské dráhy, which is a Czech railway company. While I was eating my meal and thought, that the food quality was simply bad – even worse than the food quality in the restaurant cars of the German railway, the people at the table next to me talked about how the Czech railway company prepared the food by hand and how much better it was compared to the defrosted food served by the German railway. Strange. You can paint your own picture now, but I would avoid eating in Czech restaurant cars, if I were you. Kozel ?erný seems to be worth a try, though.