Allen Wittenauer wrote:
A bit more specific:
At Yahoo!, we had either every server as a DNS slave or a DNS caching
server.
In the case of LinkedIn, we're running Solaris so nscd is significantly
better than its Linux counterpart. However, we still seem to be blowing out
the cache too much. So we'll likely switch to DNS caching servers here as
well.
the standard hadoop scripts don't tune DNS caching in the JVM, so Hadoop
doesn't notice DNS entries changing; that adds extra complexity to the
DNS-lookup-failure class of bugs -the situation where the TT and forked
jobs see different IP addresses for the same hosts
