Re: DNS caches that support partitioning ?

2012-10-14 Thread Florian Weimer
* John Levine: Are there DNS caches that allow you to partition the cache for subtrees of DNS names? That is, you can say that all entries from say, in-addr.arpa, are limited to 20% of the cache. You can build something like that using forwarders and most DNS caches. But it won't result in

Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Patrick W. Gilmore
While I hesitate to argue DNS with Mark, I feel this needs a response. On Aug 19, 2012, at 17:37 , Mark Andrews ma...@isc.org wrote: In message ddf607b5-415b-41e8-9222-eb549d3db...@semihuman.com, Chris Woodfield writes: What Patrick said. For large sites that offer services in multiple data

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Dobbins, Roland
On Aug 20, 2012, at 5:24 PM, Patrick W. Gilmore wrote: But I do not think returning multiple A records for multiple datacenters is as useful as lowering the TTL. Some folks do this via various GSLB mechanisms which selectively respond with different records based on the assumed relative

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Patrick W. Gilmore
On Aug 20, 2012, at 06:49 , Dobbins, Roland rdobb...@arbor.net wrote: On Aug 20, 2012, at 5:24 PM, Patrick W. Gilmore wrote: But I do not think returning multiple A records for multiple datacenters is as useful as lowering the TTL. Some folks do this via various GSLB mechanisms which

Re: DNS caches that support partitioning ?

2012-08-20 Thread Tony Finch
Raymond Dijkxhoorn raym...@prolocation.net wrote: When you use forwarding it doesnt cache the entry. ('forward only' option in bind for example). That's incorrect. Try configuring a forwarded zone and observe the TTLs you get in responses. The forward only option disables recursion but not

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Chris Adams
Once upon a time, Patrick W. Gilmore patr...@ianai.net said: * How many applications are even aware multiple addresses were returned? Most anything that supports IPv6 should handle this correctly, since getaddrinfo() will return a list of addresses to try. * How do you guarantee sub-second

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Patrick W. Gilmore
On Aug 20, 2012, at 08:25 , Tony Finch d...@dotat.at wrote: Patrick W. Gilmore patr...@ianai.net wrote: On Aug 19, 2012, at 17:37 , Mark Andrews ma...@isc.org wrote: Which is why the DNS supports multiple address records. Clients don't have to wait a minutes to fallover to a second

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Patrick W. Gilmore
On Aug 20, 2012, at 08:47 , Chris Adams cmad...@hiwaay.net wrote: Once upon a time, Patrick W. Gilmore patr...@ianai.net said: * How many applications are even aware multiple addresses were returned? Most anything that supports IPv6 should handle this correctly, since getaddrinfo() will

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Dobbins, Roland
On Aug 20, 2012, at 5:56 PM, Patrick W. Gilmore wrote: My question above is asking Mark how you guarantee the user/application selects the A record closest to them and only use the other A record when the closer one is unavailable. I understand - my point was that folks using a GSLB-type

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Tony Finch
Patrick W. Gilmore patr...@ianai.net wrote: On Aug 20, 2012, at 08:47 , Chris Adams cmad...@hiwaay.net wrote: Most anything that supports IPv6 should handle this correctly, since getaddrinfo() will return a list of addresses to try. Ah, the amazing new call which destroys any possibility

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Shumon Huque
On 8/20/12 10:11 AM, Tony Finch wrote: Patrick W. Gilmore patr...@ianai.net wrote: On Aug 20, 2012, at 08:47 , Chris Adams cmad...@hiwaay.net wrote: Most anything that supports IPv6 should handle this correctly, since getaddrinfo() will return a list of addresses to try. Ah, the amazing new

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Nick Hilliard
On 20/08/2012 14:18, Patrick W. Gilmore wrote: On Aug 20, 2012, at 08:47 , Chris Adams cmad...@hiwaay.net wrote: Most anything that supports IPv6 should handle this correctly, since getaddrinfo() will return a list of addresses to try. Ah, the amazing new call which destroys any possibility

Re: DNS caches that support partitioning ?

2012-08-20 Thread Aaron Hopkins
On Sat, 18 Aug 2012, Patrick W. Gilmore wrote: IMHO, if Google losses a datacenter and all users are stuck waiting for a long TTL to run out, that is Very Bad. In fact, I would call even 2.5 minutes (average of 5 min TTL) Very Bad. I'm impressed they are comfortable with a 300 second TTL.

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Tony Finch
Shumon Huque shu...@upenn.edu wrote: On 8/20/12 10:11 AM, Tony Finch wrote: The problem is RFC 3484 address selection; getaddrinfo is just the usual place this is implemented. I had believed that there was work in progress to fix this problem with the specs but it seems to have stalled.

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Tim Chown
On 20 Aug 2012, at 16:39, Tony Finch d...@dotat.at wrote: Shumon Huque shu...@upenn.edu wrote: On 8/20/12 10:11 AM, Tony Finch wrote: The problem is RFC 3484 address selection; getaddrinfo is just the usual place this is implemented. I had believed that there was work in progress to fix

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Patrick W. Gilmore
On Aug 20, 2012, at 10:07 , Dobbins, Roland rdobb...@arbor.net wrote: On Aug 20, 2012, at 5:56 PM, Patrick W. Gilmore wrote: My question above is asking Mark how you guarantee the user/application selects the A record closest to them and only use the other A record when the closer one is

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Mark Andrews
In message 0d919d57-bda0-4fda-873d-3dc0cd574...@ianai.net, Patrick W. Gilmore writes: On Aug 20, 2012, at 06:49 , Dobbins, Roland rdobb...@arbor.net = wrote: On Aug 20, 2012, at 5:24 PM, Patrick W. Gilmore wrote: =20 But I do not think returning multiple A records for multiple =

Re: Return two locations or low TTL [was: DNS caches that support partitioning ?]

2012-08-20 Thread Mark Andrews
In message 20120820124734.ga14...@hiwaay.net, Chris Adams writes: Once upon a time, Patrick W. Gilmore patr...@ianai.net said: * How many applications are even aware multiple addresses were returned? Most anything that supports IPv6 should handle this correctly, since getaddrinfo() will

Re: DNS caches that support partitioning ?

2012-08-19 Thread Chris Woodfield
What Patrick said. For large sites that offer services in multiple data centers on multiple IPs that can individually fail at any time, 300 seconds is actually a bit on the long end. -C On Aug 18, 2012, at 3:43 PM, Patrick W. Gilmore patr...@ianai.net wrote: On Aug 18, 2012, at 8:44, Jimmy

Re: DNS caches that support partitioning ?

2012-08-19 Thread Mark Andrews
In message ddf607b5-415b-41e8-9222-eb549d3db...@semihuman.com, Chris Woodfiel d writes: What Patrick said. For large sites that offer services in multiple data = centers on multiple IPs that can individually fail at any time, 300 = seconds is actually a bit on the long end. -C Which is why

Re: DNS caches that support partitioning ?

2012-08-19 Thread William Herrin
On Sun, Aug 19, 2012 at 5:37 PM, Mark Andrews ma...@isc.org wrote: As for the original problem. LRU replacement will keep hot items in the cache unless it is seriously undersized. Maybe. This discussion is reminiscent of the Linux swappiness debate. Early in the 2.x series Linux kernels, the

Re: DNS caches that support partitioning ?

2012-08-19 Thread Jimmy Hess
On 8/19/12, Mark Andrews ma...@isc.org wrote: As for the original problem. LRU replacement will keep hot items in the cache unless it is seriously undersized. [snip] Well, that's the problem. Items that are not relatively hot will be purged, even though they may be very popular RRs. Cache

Re: DNS caches that support partitioning ?

2012-08-19 Thread Gary Buhrmaster
Re: LRU badness One approach is called adaptive replacement cache (ARC) which is used by Oracle/Sun in ZFS, and was used in PostgreSQL for a time (and slightly modified to (as I recall) to be more like 2Q due to concerns over the IBM patent on the algorithm). Unfortunately, we do not have any

Re: DNS caches that support partitioning ?

2012-08-18 Thread Raymond Dijkxhoorn
Hi! The cache needs to be big enough that it has a thrashy bit that is getting changed all the time. Those are the records that go into the cache and then die without being queried again. If the problem is that there's some other record in there that might be queried again, but that doesn't

Re: DNS caches that support partitioning ?

2012-08-18 Thread Jimmy Hess
On 8/17/12, Michael Thomas m...@mtcc.com wrote: If the dnsbl queries are not likely to be used again, why don't they set their ttl way down? Because the DNSBLs don't tune the TTLs for individual responses; they likely still benefit from extended caching, high TTLs for responses makes sense for

Re: DNS caches that support partitioning ?

2012-08-18 Thread Patrick W. Gilmore
On Aug 18, 2012, at 5:35, Raymond Dijkxhoorn raym...@prolocation.net wrote: Reverse DNS isnt the only issue here. There are many sites that give each user a subdomain. And if i look at my top talkers on some busy resolvers i do see that thats doing about 25-30% of the lookups currently.

DNS caches that support partitioning ?

2012-08-17 Thread John Levine
Are there DNS caches that allow you to partition the cache for subtrees of DNS names? That is, you can say that all entries from say, in-addr.arpa, are limited to 20% of the cache. The application I have in mind is to see if it helps to keep DNSBL traffic, which caches poorly, from pushing other

Re: DNS caches that support partitioning ?

2012-08-17 Thread Andrew Sullivan
On Fri, Aug 17, 2012 at 04:13:09PM -, John Levine wrote: The application I have in mind is to see if it helps to keep DNSBL traffic, which caches poorly, from pushing other stuff out of the cache, but there are doubtless others. If it's getting evicted from cache because other things are

Re: DNS caches that support partitioning ?

2012-08-17 Thread valdis . kletnieks
On Fri, 17 Aug 2012 15:32:11 -0400, Andrew Sullivan said: On Fri, Aug 17, 2012 at 04:13:09PM -, John Levine wrote: The application I have in mind is to see if it helps to keep DNSBL traffic, which caches poorly, from pushing other stuff out of the cache, but there are doubtless others.

Re: DNS caches that support partitioning ?

2012-08-17 Thread Michael Thomas
On 08/17/2012 01:32 PM, valdis.kletni...@vt.edu wrote: On Fri, 17 Aug 2012 15:32:11 -0400, Andrew Sullivan said: On Fri, Aug 17, 2012 at 04:13:09PM -, John Levine wrote: The application I have in mind is to see if it helps to keep DNSBL traffic, which caches poorly, from pushing other

Re: DNS caches that support partitioning ?

2012-08-17 Thread Andrew Sullivan
On Fri, Aug 17, 2012 at 04:32:45PM -0400, valdis.kletni...@vt.edu wrote: I think John's issue is that he's seeing those other queries *not* benefiting from the caching because they get pushed out by DNSBL queries that will likely not ever be used again. You don't want your cached entry for

Re: DNS caches that support partitioning ?

2012-08-17 Thread John Levine
The cache needs to be big enough that it has a thrashy bit that is getting changed all the time. Those are the records that go into the cache and then die without being queried again. If the problem is that there's some other record in there that might be queried again, but that doesn't get