Re: bind-9.10.0-P2 memory leak?
On Mon, 13 Oct 2014, Thomas Schulz wrote: I restarted bind 9.9.6 with a max-cache-size of 30M. We have 3 views. The inital process size was 36 MB. The process grew to 184 MB. It grew to 596 MB without the max-cache-size being set and was still growing when I restarted it. BUT when I now do an rndc dumpdb -cache, the named_dump.db file contains only the line ; Dump complete and nothing else. So, if you put any limit on the cache size, you will end up with an empty cache. I do believe that there is a bug that needs to be fixed. I wasn't able to reproduce this with 9.9.6 (or a recent master). Can you please send your configuration (like named-checkconf -px) to bind9-bugs AT isc.org? Thank you. I sent the configuration, hopefully in a usable format. Note that my concern is not specifically with the cache. I was investigating the cache as a possable cause of the unlimited growth in memory usage that I am seeing. The various experiments that I have done seem to point to the cache as the cause of the problem. See also the posts with the subject 'Process size versus cache size'. To the originator of this thread. Sorry if I have hijacked your thread. It is possible that the problem you are seeing has a different cause than the problem I am seeing. Perhaps you should also send in a report to bind9-bugs AT isc.org. Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
... Heh thanks, yeah...initially I was erring on the side of caution and using 9.9.x because it's served us well (~20k recursive clients without any significant problems). Meanwhile we've been keeping a close eye on community comments, and to be honest opinions wax and wane. Just as I think it's stabilized, someone else complains. I suppose sticking to 9.9.x a bit longer is wise. That said, based on the 9.10.1 fixes, we will run it through our own perf tests for comparison. Upgrades are automated and easy, but I'd obviously like to go live with the latest version unless there is a strong technical reason otherwise. FYI, 9.9 is the current Extended Support Version (ESV). If you're looking for a version of BIND with a long period of maintenance, there will be ongoing 9.9.x, 9.9.x+1 etc. releases and interim patches if needed. http://www.isc.org/downloads/software-support-policy/ I mentioned this earlier, but I have been seeing the very large increases in process size with Bind 9.9.5-P1 and 9.9.6b1. I have just installed 9.10.1rc2 on one of our secondary name servers. In time I will be able to see if 9.10.1rc2 shows a bigger increase in process size than 9.9.5-P1 did. I have restarted 9.9.6b1 with max-cache-size 30M on our primary server. Both experiments will take some time before I can tell what is happening. For those seeing this problem on bind 9.10.1, did you upgrade from 9.9.6 or from an earlier version of bind 9.9.*? As mentioned above, I am seeing this problem on 9.9.6. I do not find bind 9.10.1 growing any faster than 9.9.6 does. I restarted bind 9.9.6 with a max-cache-size of 30M. We have 3 views. The inital process size was 36 MB. The process grew to 184 MB. It grew to 596 MB without the max-cache-size being set and was still growing when I restarted it. BUT when I now do an rndc dumpdb -cache, the named_dump.db file contains only the line ; Dump complete and nothing else. So, if you put any limit on the cache size, you will end up with an empty cache. I do believe that there is a bug that needs to be fixed. Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
On Monday 13/10/2014 at 1:32 pm, sch...@adi.com (Thomas Schulz) wrote: ... Heh thanks, yeah...initially I was erring on the side of caution and using 9.9.x because it's served us well (~20k recursive clients without any significant problems). Meanwhile we've been keeping a close eye on community comments, and to be honest opinions wax and wane. Just as I think it's stabilized, someone else complains. I suppose sticking to 9.9.x a bit longer is wise. That said, based on the 9.10.1 fixes, we will run it through our own perf tests for comparison. Upgrades are automated and easy, but I'd obviously like to go live with the latest version unless there is a strong technical reason otherwise. FYI, 9.9 is the current Extended Support Version (ESV). If you're looking for a version of BIND with a long period of maintenance, there will be ongoing 9.9.x, 9.9.x+1 etc. releases and interim patches if needed. http://www.isc.org/downloads/software-support-policy/ I mentioned this earlier, but I have been seeing the very large increases in process size with Bind 9.9.5-P1 and 9.9.6b1. I have just installed 9.10.1rc2 on one of our secondary name servers. In time I will be able to see if 9.10.1rc2 shows a bigger increase in process size than 9.9.5-P1 did. I have restarted 9.9.6b1 with max-cache-size 30M on our primary server. Both experiments will take some time before I can tell what is happening. For those seeing this problem on bind 9.10.1, did you upgrade from 9.9.6 or from an earlier version of bind 9.9.*? As mentioned above, I am seeing this problem on 9.9.6. I do not find bind 9.10.1 growing any faster than 9.9.6 does. I restarted bind 9.9.6 with a max-cache-size of 30M. We have 3 views. The inital process size was 36 MB. The process grew to 184 MB. It grew to 596 MB without the max-cache-size being set and was still growing when I restarted it. BUT when I now do an rndc dumpdb -cache, the named_dump.db file contains only the line ; Dump complete and nothing else. So, if you put any limit on the cache size, you will end up with an empty cache. I do believe that there is a bug that needs to be fixed. With Freebsd 10.0, I tried the 9.10 leak work around by with max-cache-size, and it didn't stop the named memory foot print from growing to 2+ GB. I need 9.10 for the its white listing of RPZ hits. I'm building a new Freebsd 9.3 VM and bind 9.10. (vmtools doesn't support fbsd 10) Len ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
On Mon, 13 Oct 2014, Thomas Schulz wrote: I restarted bind 9.9.6 with a max-cache-size of 30M. We have 3 views. The inital process size was 36 MB. The process grew to 184 MB. It grew to 596 MB without the max-cache-size being set and was still growing when I restarted it. BUT when I now do an rndc dumpdb -cache, the named_dump.db file contains only the line ; Dump complete and nothing else. So, if you put any limit on the cache size, you will end up with an empty cache. I do believe that there is a bug that needs to be fixed. I wasn't able to reproduce this with 9.9.6 (or a recent master). Can you please send your configuration (like named-checkconf -px) to bind9-bugs AT isc.org? Thank you. ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
Hi Thomas On Mon, Oct 13, 2014 at 02:31:37PM -0400, Thomas Schulz wrote: I restarted bind 9.9.6 with a max-cache-size of 30M. We have 3 views. The inital process size was 36 MB. The process grew to 184 MB. It grew to 596 MB without the max-cache-size being set and was still growing when I restarted it. BUT when I now do an rndc dumpdb -cache, the max-cache-size affects the size of an internal memory context that's used for cache. But the process size is made up of all the memory the program uses (its various memory contexts, memory used outside contexts such as by libraries used by BIND, direct allocations, BSS and other maps). If you want to check how much memory is used by the cache, and if it goes above max-cache-size, enable the statistics channels in the named configuration and dump XML statistics to look at the usage for the cache contexts. If you have max-cache-size set, any unchecked growth above it of the cache contexts is a bug. But don't compare max-cache-size against the process size. Other parts of the process apart from the cache use memory too. Also as Jeremy asked, you may want to share your named configuration with us so we can check it. Mukund pgpGvc4x_Tf1_.pgp Description: PGP signature ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Re: bind-9.10.0-P2 memory leak?
___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
... Heh thanks, yeah...initially I was erring on the side of caution and using 9.9.x because it's served us well (~20k recursive clients without any significant problems). Meanwhile we've been keeping a close eye on community comments, and to be honest opinions wax and wane. Just as I think it's stabilized, someone else complains. I suppose sticking to 9.9.x a bit longer is wise. That said, based on the 9.10.1 fixes, we will run it through our own perf tests for comparison. Upgrades are automated and easy, but I'd obviously like to go live with the latest version unless there is a strong technical reason otherwise. FYI, 9.9 is the current Extended Support Version (ESV). If you're looking for a version of BIND with a long period of maintenance, there will be ongoing 9.9.x, 9.9.x+1 etc. releases and interim patches if needed. http://www.isc.org/downloads/software-support-policy/ ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
... Heh thanks, yeah...initially I was erring on the side of caution and using 9.9.x because it's served us well (~20k recursive clients without any significant problems). Meanwhile we've been keeping a close eye on community comments, and to be honest opinions wax and wane. Just as I think it's stabilized, someone else complains. I suppose sticking to 9.9.x a bit longer is wise. That said, based on the 9.10.1 fixes, we will run it through our own perf tests for comparison. Upgrades are automated and easy, but I'd obviously like to go live with the latest version unless there is a strong technical reason otherwise. FYI, 9.9 is the current Extended Support Version (ESV). If you're looking for a version of BIND with a long period of maintenance, there will be ongoing 9.9.x, 9.9.x+1 etc. releases and interim patches if needed. http://www.isc.org/downloads/software-support-policy/ I mentioned this earlier, but I have been seeing the very large increases in process size with Bind 9.9.5-P1 and 9.9.6b1. I have just installed 9.10.1rc2 on one of our secondary name servers. In time I will be able to see if 9.10.1rc2 shows a bigger increase in process size than 9.9.5-P1 did. I have restarted 9.9.6b1 with max-cache-size 30M on our primary server. Both experiments will take some time before I can tell what is happening. Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
On Tue, 9 Sep 2014, Thomas Schulz wrote: What version did you upgrade from? I am seeing bind 9.9.5 and 9.9.6 grow without any evidence that it will ever stop. See my mail to this list with the subject Re: Process size versus cache size. Mine is growing slower than yours, but it is now up to 548 MB. Can you copy and paste the out of memory error you are seeing? Is it still growing? Does it appear to work? ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
Mike Hoskins wrote: Do you guys have max-cache-size set? I didn't see it in the borderworlds named.conf. I've seen similar growth problems when testing 9.x before setting that (experiment at the time just to see what would happen, and confirmed this behavior). Set sensible resource limits based on available resources. I am going to see what happens with max-cache-size set, but I am convinced that there is a bug in bind. My named has been running for 7.5 weeks now and has been steadily growing in size except for a 1.5 week pause after I did an rndc flush. The process size started out at 36 MB and is now up to 584 MB. But when I do an rndc dumpdb -cache I get a file that is only 5 MB in size. Given the automatic cache cleaning, named should stabilize in size in less than 7.5 weeks. -Original Message- From: Vin?cius Ferr?o fer...@if.ufrj.br Date: Tuesday, September 9, 2014 at 10:17 AM To: Thomas Schulz sch...@adi.com Cc: bind-us...@isc.org bind-us...@isc.org Subject: Re: bind-9.10.0-P2 memory leak? I'm having the exactly same issue. Take a look at my post @ServerFault: http://serverfault.com/questions/616752/bind-9-10-constantly-killed-on-fre ebsd-10-0-with-out-of-swap-space Sent from my iPhone On 09/09/2014, at 11:15, Thomas Schulz sch...@adi.com wrote: Hello I recently upgraded my authoritative nameservers to bind-9.10.0-P2 and after a while one of them ended up using all its swap and the named process got killed. The other servers are seeing similar behaviour, but I restarted named on all of them to postpone further crashes. I am using rate-limiting as well DLZ with PostgreSQL. The server has two views. The operating system is FreeBSD 8.4. My configuration: http://borderworlds.dk/~xi/named-leak/named.conf Log of the memory usage: http://borderworlds.dk/~xi/named-leak/named-mem-usage.log As you can see, in less than a week, named has grown more than 900MB in size. Is anyone else experiencing something similar? If I need to provide more information, I will be happy to do so. -- Christian Laursen What version did you upgrade from? I am seeing bind 9.9.5 and 9.9.6 grow without any evidence that it will ever stop. See my mail to this list with the subject Re: Process size versus cache size. Mine is growing slower than yours, but it is now up to 548 MB. Tom Schulz Applied Dynamics Intl. sch...@adi.com Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
Can you copy and paste the out of memory error you are seeing? Is it still growing? Does it appear to work? I see your other thread answers some. https://lists.isc.org/pipermail/bind-users/2014-July/093618.html ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
-Original Message- From: Thomas Schulz sch...@adi.com Date: Friday, September 12, 2014 at 11:47 AM To: bind-us...@isc.org bind-us...@isc.org Subject: Re: bind-9.10.0-P2 memory leak? Mike Hoskins wrote: Do you guys have max-cache-size set? I didn't see it in the borderworlds named.conf. I've seen similar growth problems when testing 9.x before setting that (experiment at the time just to see what would happen, and confirmed this behavior). Set sensible resource limits based on available resources. I am going to see what happens with max-cache-size set, but I am convinced that there is a bug in bind. My named has been running for 7.5 weeks now and has been steadily growing in size except for a 1.5 week pause after I did an rndc flush. The process size started out at 36 MB and is now up to 584 MB. But when I do an rndc dumpdb -cache I get a file that is only 5 MB in size. Given the automatic cache cleaning, named should stabilize in size in less than 7.5 weeks. Just to be clear, I tend to agree with the memory leak hypothesis at this point... Based on the described behavior and past experience I related, I initially just did a search of your config looking for max-cache-size. Sorry for that, was in training at the time and somewhat distracted. However, your use case is obviously very different from mine as you are not doing recursion (my test environment without max-cache-size was, and getting hit with an almost endless stream of random real-world queries from my queryfile). That said, I wonder if it could be dlz related? That's the only thing I see special about your config. Just trying to find possible clues, since I have ran all 9.9.x versions over time in heavily loaded production environments (authoritative and recursive) without seeing the unbounded growth you mentioned below for 9.9.x. I do have a lot of interest in the community getting to the bottom of this, as we are just planning a large upgrade in one of our environments which will move caching clusters serving 6-8k clients over to 9.10.1. -Original Message- From: Vinícius Ferrão fer...@if.ufrj.br Date: Tuesday, September 9, 2014 at 10:17 AM To: Thomas Schulz sch...@adi.com Cc: bind-us...@isc.org bind-us...@isc.org Subject: Re: bind-9.10.0-P2 memory leak? I'm having the exactly same issue. Take a look at my post @ServerFault: http://serverfault.com/questions/616752/bind-9-10-constantly-killed-on-f re ebsd-10-0-with-out-of-swap-space Sent from my iPhone On 09/09/2014, at 11:15, Thomas Schulz sch...@adi.com wrote: Hello I recently upgraded my authoritative nameservers to bind-9.10.0-P2 and after a while one of them ended up using all its swap and the named process got killed. The other servers are seeing similar behaviour, but I restarted named on all of them to postpone further crashes. I am using rate-limiting as well DLZ with PostgreSQL. The server has two views. The operating system is FreeBSD 8.4. My configuration: http://borderworlds.dk/~xi/named-leak/named.conf Log of the memory usage: http://borderworlds.dk/~xi/named-leak/named-mem-usage.log As you can see, in less than a week, named has grown more than 900MB in size. Is anyone else experiencing something similar? If I need to provide more information, I will be happy to do so. -- Christian Laursen What version did you upgrade from? I am seeing bind 9.9.5 and 9.9.6 grow without any evidence that it will ever stop. See my mail to this list with the subject Re: Process size versus cache size. Mine is growing slower than yours, but it is now up to 548 MB. Tom Schulz Applied Dynamics Intl. sch...@adi.com Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
On 9/12/14 11:07 AM, Mike Hoskins (michoski) wrote: I do have a lot of interest in the community getting to the bottom of this, as we are just planning a large upgrade in one of our environments which will move caching clusters serving 6-8k clients over to 9.10.1. Given all of the problems that have been reported with 9.10 you may wish to reconsider that plan. Doug ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
Can you copy and paste the out of memory error you are seeing? Is it still growing? Does it appear to work? I see your other thread answers some. https://lists.isc.org/pipermail/bind-users/2014-July/093618.html Unfortunately the logs containing the out of memory errors have been purged. Those errors have not reoccurred with the 64 bit named. Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
-Original Message- From: Doug Barton do...@dougbarton.us Date: Friday, September 12, 2014 at 2:15 PM To: Mike Hoskins micho...@cisco.com, Thomas Schulz sch...@adi.com, bind-us...@isc.org bind-us...@isc.org Subject: Re: bind-9.10.0-P2 memory leak? On 9/12/14 11:07 AM, Mike Hoskins (michoski) wrote: I do have a lot of interest in the community getting to the bottom of this, as we are just planning a large upgrade in one of our environments which will move caching clusters serving 6-8k clients over to 9.10.1. Given all of the problems that have been reported with 9.10 you may wish to reconsider that plan. Heh thanks, yeah...initially I was erring on the side of caution and using 9.9.x because it's served us well (~20k recursive clients without any significant problems). Meanwhile we've been keeping a close eye on community comments, and to be honest opinions wax and wane. Just as I think it's stabilized, someone else complains. I suppose sticking to 9.9.x a bit longer is wise. That said, based on the 9.10.1 fixes, we will run it through our own perf tests for comparison. Upgrades are automated and easy, but I'd obviously like to go live with the latest version unless there is a strong technical reason otherwise. ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
bind-9.10.0-P2 memory leak?
Hello I recently upgraded my authoritative nameservers to bind-9.10.0-P2 and after a while one of them ended up using all its swap and the named process got killed. The other servers are seeing similar behaviour, but I restarted named on all of them to postpone further crashes. I am using rate-limiting as well DLZ with PostgreSQL. The server has two views. The operating system is FreeBSD 8.4. My configuration: http://borderworlds.dk/~xi/named-leak/named.conf Log of the memory usage: http://borderworlds.dk/~xi/named-leak/named-mem-usage.log As you can see, in less than a week, named has grown more than 900MB in size. Is anyone else experiencing something similar? If I need to provide more information, I will be happy to do so. -- Christian Laursen ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
Hello I recently upgraded my authoritative nameservers to bind-9.10.0-P2 and after a while one of them ended up using all its swap and the named process got killed. The other servers are seeing similar behaviour, but I restarted named on all of them to postpone further crashes. I am using rate-limiting as well DLZ with PostgreSQL. The server has two views. The operating system is FreeBSD 8.4. My configuration: http://borderworlds.dk/~xi/named-leak/named.conf Log of the memory usage: http://borderworlds.dk/~xi/named-leak/named-mem-usage.log As you can see, in less than a week, named has grown more than 900MB in size. Is anyone else experiencing something similar? If I need to provide more information, I will be happy to do so. -- Christian Laursen What version did you upgrade from? I am seeing bind 9.9.5 and 9.9.6 grow without any evidence that it will ever stop. See my mail to this list with the subject Re: Process size versus cache size. Mine is growing slower than yours, but it is now up to 548 MB. Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
I'm having the exactly same issue. Take a look at my post @ServerFault: http://serverfault.com/questions/616752/bind-9-10-constantly-killed-on-freebsd-10-0-with-out-of-swap-space Sent from my iPhone On 09/09/2014, at 11:15, Thomas Schulz sch...@adi.com wrote: Hello I recently upgraded my authoritative nameservers to bind-9.10.0-P2 and after a while one of them ended up using all its swap and the named process got killed. The other servers are seeing similar behaviour, but I restarted named on all of them to postpone further crashes. I am using rate-limiting as well DLZ with PostgreSQL. The server has two views. The operating system is FreeBSD 8.4. My configuration: http://borderworlds.dk/~xi/named-leak/named.conf Log of the memory usage: http://borderworlds.dk/~xi/named-leak/named-mem-usage.log As you can see, in less than a week, named has grown more than 900MB in size. Is anyone else experiencing something similar? If I need to provide more information, I will be happy to do so. -- Christian Laursen What version did you upgrade from? I am seeing bind 9.9.5 and 9.9.6 grow without any evidence that it will ever stop. See my mail to this list with the subject Re: Process size versus cache size. Mine is growing slower than yours, but it is now up to 548 MB. Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
Do you guys have max-cache-size set? I didn't see it in the borderworlds named.conf. I've seen similar growth problems when testing 9.x before setting that (experiment at the time just to see what would happen, and confirmed this behavior). Set sensible resource limits based on available resources. -Original Message- From: Vinícius Ferrão fer...@if.ufrj.br Date: Tuesday, September 9, 2014 at 10:17 AM To: Thomas Schulz sch...@adi.com Cc: bind-us...@isc.org bind-us...@isc.org Subject: Re: bind-9.10.0-P2 memory leak? I'm having the exactly same issue. Take a look at my post @ServerFault: http://serverfault.com/questions/616752/bind-9-10-constantly-killed-on-fre ebsd-10-0-with-out-of-swap-space Sent from my iPhone On 09/09/2014, at 11:15, Thomas Schulz sch...@adi.com wrote: Hello I recently upgraded my authoritative nameservers to bind-9.10.0-P2 and after a while one of them ended up using all its swap and the named process got killed. The other servers are seeing similar behaviour, but I restarted named on all of them to postpone further crashes. I am using rate-limiting as well DLZ with PostgreSQL. The server has two views. The operating system is FreeBSD 8.4. My configuration: http://borderworlds.dk/~xi/named-leak/named.conf Log of the memory usage: http://borderworlds.dk/~xi/named-leak/named-mem-usage.log As you can see, in less than a week, named has grown more than 900MB in size. Is anyone else experiencing something similar? If I need to provide more information, I will be happy to do so. -- Christian Laursen What version did you upgrade from? I am seeing bind 9.9.5 and 9.9.6 grow without any evidence that it will ever stop. See my mail to this list with the subject Re: Process size versus cache size. Mine is growing slower than yours, but it is now up to 548 MB. Tom Schulz Applied Dynamics Intl. sch...@adi.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
On 9/9/2014 05:05, lcon...@go2france.com wrote: freebsd 10.0, bind-9.10.0-p2 logging the rss field for named process: less /var/tmp/bind_rss_history.txt This never happened with earlier BIND9, and our mx1 uses this recursive BIND machine for all domain/ptr lookups I've never seen any bind take over 1GB of RAM. max-cache-size isn't the solution, only a band-aid the sawtooth above is from restarting named. named has halted twice in the past couple weeks, we suspected some kind of attack, the only trace we had was in syslog with something like swap space failed, named halted, but with a dedicated DNS box and 3 GB, there should never be any swapping. I set a watcher for swap used 1%. Got an alert, I saw the named rss to be 1.9GB. restarted bind and wrote the rss named logging script. Len This is a bit worrying for me, as I am running this version on my master. Do you mind sharing the rss watcher/logging script? -- staticsafe https://staticsafe.ca ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
At 09:40 PM 9/9/2014, you wrote:
On 9/9/2014 05:05, lcon...@go2france.com wrote:
freebsd 10.0, bind-9.10.0-p2
logging the rss field for named process:
less /var/tmp/bind_rss_history.txt
This never happened with earlier BIND9, and our mx1 uses this recursive
BIND machine for all domain/ptr lookups
I've never seen any bind take over 1GB of RAM.
max-cache-size isn't the solution, only a band-aid
the sawtooth above is from restarting named.
named has halted twice in the past couple weeks, we suspected some kind
of attack, the only trace we had was in syslog with something like swap
space failed, named halted, but with a dedicated DNS box and 3 GB,
there should never be any swapping. I set a watcher for swap used
1%. Got an alert, I saw the named rss to be 1.9GB. restarted bind and
wrote the rss named logging script.
Len
This is a bit worrying for me, as I am running this version on my
master. Do you mind sharing the rss watcher/logging script?
cat /usr/local/bin/bind_rss_history.sh
#!/bin/sh
touch /var/tmp/bind_rss_history.txt
RSS=`ps auxw | awk '/^bind.*named/{print $6}'`
NOW=`date +%Y-%m-%d %H:%M:%S`
echo $NOW $RSS | awk '{printf %10s%10s%11s\n,$1,$2,$3}'
/var/tmp/bind_rss_history.txt
exit 0
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
from this list
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Re: bind-9.10.0-P2 memory leak?
On 9/9/2014 23:17, Len Conrad wrote:
cat /usr/local/bin/bind_rss_history.sh
#!/bin/sh
touch /var/tmp/bind_rss_history.txt
RSS=`ps auxw | awk '/^bind.*named/{print $6}'`
NOW=`date +%Y-%m-%d %H:%M:%S`
echo $NOW $RSS | awk '{printf %10s%10s%11s\n,$1,$2,$3}'
/var/tmp/bind_rss_history.txt
exit 0
Thanks!
--
staticsafe
https://staticsafe.ca
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
from this list
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
