Re: [Pdns-users] Possible bug observed in PowerDNS Recursor 3.2.1
On Thu, 2010-08-05 at 11:55 +0100, K Storbeck wrote: Hello all, We've been experiencing the problem too, on 3.2. NB: As far as I know, there is no 3.2.1 version: http://downloads.powerdns.com/releases/ does not list such a version. Stop assuming it exists :) You're right, my bad, I installed it using the RPM and misread the version number when I reported this. :-) -- Nuno Nunes (nuno.nu...@optimus.pt) Tel: 351931003485 | Fax: 351931023485 Edifício Optimus Av. D. João II - Lt. 1.06.2.4 1990-095 Lisboa Portugal ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Possible bug observed in PowerDNS Recursor 3.2.1
On 8/4/2010 6:36 AM, Nuno Nunes wrote: Hello all, I've gone through the last few months of the ML, up until the announcement of the release of 3.2.1, and didn't find any reference to this bug I'm apparently seeing, so I'm reporting this to you all for help. I work at an ISP where we have a number of servers running PowerDNS Resolver 3.2.1 as our customer-facing resolvers. We have had this setup for a few months now and sometimes a weird thing happens (and no, I can't reproduce it in any deterministic way and it only happens sometimes): when the TTL for a record of a given zone expires and a new request comes in for it, some of the caches on the farm go out and get the new information, but some others just seem to ignore the TTL and stick with the old data forever. This is most notable when a zone changes name servers and the owner of the zone comes complaining to us that we still have the old data, even after the appropriate amount of time has elapsed for it to have been refreshed (and on these cases we typically observe this behaviour on NS records, but we have observed it on A records also, for example). I see this all the time on BIND resolvers. The keys to the situation are: * Domain's old NS records have a relatively long TTL (from old auth. servers) * Domain owner changes auth. servers with registrar * Domain owner does NOT update data on old auth. servers. (they're now serving stale data, but authoritatively) Since the domain owner is your ISP customer, you get get queries for the domain relatively often, so your recursive servers rely on the cached NS records for the domain (the ones that point to the auth. server serving stale data). I think that BIND resets the TTL when the recursive server sees NS records in the authority section of a response. Maybe PowerDNS is doing this as well? I generally advise the domian owner to have the domain removed from the old auth. server. -- Dave ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Possible bug observed in PowerDNS Recursor 3.2.1
Briefly diving into this: On Thu, Aug 05, 2010 at 10:12:54AM -0400, Dave Sparro wrote: I see this all the time on BIND resolvers. The keys to the situation are: * Domain's old NS records have a relatively long TTL (from old auth. servers) * Domain owner changes auth. servers with registrar * Domain owner does NOT update data on old auth. servers. (they're now serving stale data, but authoritatively) Since the domain owner is your ISP customer, you get get queries for the domain relatively often, so your recursive servers rely on the cached NS records for the domain (the ones that point to the auth. server serving stale data). I think that BIND resets the TTL when the recursive server sees NS records in the authority section of a response. Maybe PowerDNS is doing this as well? PowerDNS 3.2 has a bug in this respect where it keeps believing the old data. The 3.3 snapshot, in full production in some places, has this issue resolved. I'll trawl through the entire thread to see if this is indeed the issue we are talking about. Bert I generally advise the domian owner to have the domain removed from the old auth. server. -- Dave ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
[Pdns-users] Possible bug observed in PowerDNS Recursor 3.2.1
Hello all, I've gone through the last few months of the ML, up until the announcement of the release of 3.2.1, and didn't find any reference to this bug I'm apparently seeing, so I'm reporting this to you all for help. I work at an ISP where we have a number of servers running PowerDNS Resolver 3.2.1 as our customer-facing resolvers. We have had this setup for a few months now and sometimes a weird thing happens (and no, I can't reproduce it in any deterministic way and it only happens sometimes): when the TTL for a record of a given zone expires and a new request comes in for it, some of the caches on the farm go out and get the new information, but some others just seem to ignore the TTL and stick with the old data forever. This is most notable when a zone changes name servers and the owner of the zone comes complaining to us that we still have the old data, even after the appropriate amount of time has elapsed for it to have been refreshed (and on these cases we typically observe this behaviour on NS records, but we have observed it on A records also, for example). Now we have had this happen at least three times over the last months and we've tried to narrow it down to a specific set of circumstances, but we haven't been able to really find a pattern. What we do know is that every time this happens, some of the servers behave correctly (TTL expires = get new data) and others don't. And when that happens not even `rec_control wipe-cache` will work. The servers are all identical (same HW, same OS and same SW). Has anyone else observed something like this before? Is it a known bug and I just failed to find it being discussed? More importantly: is there a fix for this behaviour? Thanks, Nuno Nunes -- Nuno Nunes (nuno.nu...@optimus.pt) Tel: 351931003485 | Fax: 351931023485 Edifício Optimus Av. D. João II - Lt. 1.06.2.4 1990-095 Lisboa Portugal ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Possible bug observed in PowerDNS Recursor 3.2.1
On 08/04/2010 01:36 PM, Nuno Nunes wrote: Hello all, I've gone through the last few months of the ML, up until the announcement of the release of 3.2.1, and didn't find any reference to this bug I'm apparently seeing, so I'm reporting this to you all for help. I work at an ISP where we have a number of servers running PowerDNS Resolver 3.2.1 as our customer-facing resolvers. We have had this setup for a few months now and sometimes a weird thing happens (and no, I can't reproduce it in any deterministic way and it only happens sometimes): when the TTL for a record of a given zone expires and a new request comes in for it, some of the caches on the farm go out and get the new information, but some others just seem to ignore the TTL and stick with the old data forever. This is most notable when a zone changes name servers and the owner of the zone comes complaining to us that we still have the old data, even after the appropriate amount of time has elapsed for it to have been refreshed (and on these cases we typically observe this behaviour on NS records, but we have observed it on A records also, for example). Now we have had this happen at least three times over the last months and we've tried to narrow it down to a specific set of circumstances, but we haven't been able to really find a pattern. What we do know is that every time this happens, some of the servers behave correctly (TTL expires = get new data) and others don't. And when that happens not even `rec_control wipe-cache` will work. The servers are all identical (same HW, same OS and same SW). Has anyone else observed something like this before? Is it a known bug and I just failed to find it being discussed? More importantly: is there a fix for this behaviour? Indeed. I saw the exact same thing, like 3 or 4 times in the last couple of months, with the exact same simptoms. Also at an ISP, customers complaining about old records after changing nameservers for a domain. Couldn't find the cause either, although I did not investigate in detail. Good to know I'm not crazy ;) I have to look into it next time this pops up with a domain. I have no further details unfortunately. I don't think it came up until now on the list, it's pretty rare and vague to get good details on the problem. -- Imre Gergely Yahoo!: gergelyimre | ICQ#: 101510959 MSN: gergely_imre | GoogleTalk: gergelyimre gpg --keyserver subkeys.pgp.net --recv-keys 0x34525305 ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Possible bug observed in PowerDNS Recursor 3.2.1
Imre Gergely wrote: On 08/04/2010 01:36 PM, Nuno Nunes wrote: Hello all, I've gone through the last few months of the ML, up until the announcement of the release of 3.2.1, and didn't find any reference to this bug I'm apparently seeing, so I'm reporting this to you all for help. I work at an ISP where we have a number of servers running PowerDNS Resolver 3.2.1 as our customer-facing resolvers. We have had this setup for a few months now and sometimes a weird thing happens (and no, I can't reproduce it in any deterministic way and it only happens sometimes): when the TTL for a record of a given zone expires and a new request comes in for it, some of the caches on the farm go out and get the new information, but some others just seem to ignore the TTL and stick with the old data forever. This is most notable when a zone changes name servers and the owner of the zone comes complaining to us that we still have the old data, even after the appropriate amount of time has elapsed for it to have been refreshed (and on these cases we typically observe this behaviour on NS records, but we have observed it on A records also, for example). Now we have had this happen at least three times over the last months and we've tried to narrow it down to a specific set of circumstances, but we haven't been able to really find a pattern. What we do know is that every time this happens, some of the servers behave correctly (TTL expires = get new data) and others don't. And when that happens not even `rec_control wipe-cache` will work. The servers are all identical (same HW, same OS and same SW). Has anyone else observed something like this before? Is it a known bug and I just failed to find it being discussed? More importantly: is there a fix for this behaviour? Indeed. I saw the exact same thing, like 3 or 4 times in the last couple of months, with the exact same simptoms. Also at an ISP, customers complaining about old records after changing nameservers for a domain. Couldn't find the cause either, although I did not investigate in detail. Good to know I'm not crazy ;) I have to look into it next time this pops up with a domain. I have no further details unfortunately. I don't think it came up until now on the list, it's pretty rare and vague to get good details on the problem. Weird. I haven't seen it, but then I have a cron job that restarts the recursor once per week, which probably refreshes anything old at that point. -- -- Steven G. Spencer, Network Administrator KSC Corporate - The Kelly Supply Family of Companies Office 308-382-8764 Ext. 231 Mobile 308-380-7957 ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
