Re: Disable cache in bind 9.6
At Wed, 04 Feb 2009 11:51:10 +0300, Dmitry Rybin wrote: > > > max-cache-size 800M; > > > It's way too much, if this applies to all of the 50 views. > > Oh! I decrease memory to 16Mb. Okay, and according to this: : Started at Feb 3 00:51 (Now Feb 4 11:15:37) MSK : Startup mem: 890M : Cur. memory usage: 2534M the additional memory needed while running is 1644M (2534 - 890), 32.88M per view (if the #of view is 50). This seems to be a possible situation, considering other memory overhead per view. If the memory footprint is now stabilized at that point, I guess you're fine with that, right? (and you could increase max-cache-size to, e.g., 64M). --- JINMEI, Tatuya Internet Systems Consortium, Inc. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
At Wed, 04 Feb 2009 11:36:12 +0300, Dmitry Rybin wrote: > >> max-cache-size 800M; > > > > It's way too much, if this applies to all of the 50 views. > > With you patch? Total memory on server 12Gb. In case you are confused: this patch only makes named honor max-cache-size (+ some possible margin) for each view, unlike the unreleased new feature 'attach-cache'. Each of your 50 views can still consume up to 800MB, so you need to expect at least 50 * 800MB of memory footprint in the worst case. --- JINMEI, Tatuya Internet Systems Consortium, Inc. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
On 04.02.09 11:51, Dmitry Rybin wrote: > Matus UHLAR - fantomas wrote: > > > > > On 04.02.09 11:23, Dmitry Rybin wrote: > >> It's impossible, :-( over 500'000 client use bind and we must use views > >> to split load on another services. > > > > Named with JINMEI Tatuy patch: > > > max-cache-size 800M; > > > It's way too much, if this applies to all of the 50 views. > > Oh! I decrease memory to 16Mb. No, I did not write that. Please don't break quoting. > > Pardon? Split load? Do you use views to point different clients to different > > server to lower load on them? > > > > If so, you better should use DNS load balancing or some kind of HW/SW load > > balancer > > > > For first time was DNS load balancing. And after grow clients base we > can use only current scheme. We think about it, but only bind with > current configuration approach to us. Yes, but now it seems to reach its possibilities, so you should better think about changing your architecture... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Quantum mechanics: The dreams stuff is made of. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Matus UHLAR - fantomas wrote: > > On 04.02.09 11:23, Dmitry Rybin wrote: >> It's impossible, :-( over 500'000 client use bind and we must use views >> to split load on another services. > > Named with JINMEI Tatuy patch: > > max-cache-size 800M; > It's way too much, if this applies to all of the 50 views. Oh! I decrease memory to 16Mb. > > Pardon? Split load? Do you use views to point different clients to different > server to lower load on them? > > If so, you better should use DNS load balancing or some kind of HW/SW load > balancer > For first time was DNS load balancing. And after grow clients base we can use only current scheme. We think about it, but only bind with current configuration approach to us. Another variant - powerdns-recursor with LUA scripting. (in testing) ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
> >> ~50 views, > Matus UHLAR - fantomas wrote: > > can't you really lower the views count? On 04.02.09 11:23, Dmitry Rybin wrote: > It's impossible, :-( over 500'000 client use bind and we must use views > to split load on another services. Pardon? Split load? Do you use views to point different clients to different server to lower load on them? If so, you better should use DNS load balancing or some kind of HW/SW load balancer -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The early bird may get the worm, but the second mouse gets the cheese. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
JINMEI Tatuya / 神明達哉 wrote: > At Wed, 04 Feb 2009 11:23:19 +0300, > Dmitry Rybin wrote: > >> Named with JINMEI Tatuy patch: >> max-cache-size 800M; > > It's way too much, if this applies to all of the 50 views. > With you patch? Total memory on server 12Gb. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
At Wed, 04 Feb 2009 11:23:19 +0300, Dmitry Rybin wrote: > Named with JINMEI Tatuy patch: > max-cache-size 800M; It's way too much, if this applies to all of the 50 views. --- JINMEI, Tatuya Internet Systems Consortium, Inc. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Matus UHLAR - fantomas wrote: >>> and let me know if it mitigates the problem? > > On 29.01.09 22:50, Dmitry Rybin wrote: >> Oh, great work. I'll try tomorrow. Named with JINMEI Tatuy patch: max-cache-size 800M; Morning Statistic version: 9.6.0-P1 CPUs found: 8 worker threads: 8 number of zones: 1040 debug level: 0 xfers running: 0 xfers deferred: 0 soa queries in progress: 0 query logging is OFF recursive clients: 167/4900/5000 tcp clients: 0/100 server is up and running Started at Feb 3 00:51 (Now Feb 4 11:15:37) MSK Startup mem: 890M Cur. memory usage: 2534M System limit: 16G ++ Incoming Requests ++ 112510181 QUERY 550 IQUERY 42 STATUS 1043 RESERVED3 101299 NOTIFY 101 UPDATE 14 RESERVED11 ++ Incoming Queries ++ 1929 RESERVED0 75241540 A 2105214 NS 100 CNAME 276292 SOA 2490 WKS 26826476 PTR 2 HINFO 4690581 MX 236619 TXT 24 X25 2003829 17 LOC 713837 SRV 46397 NAPTR 58 A6 1022 SPF 4 IXFR 5 AXFR 317561 ANY 23 Others ++ Outgoing Queries ++ >> Yes, I try it. But I can't set ttl to 0. It didn't work. Recursive query >> fails, and authoritative query back to clients with ttl 0 :( > > Yes, that is what "Setting TTL to 0" means. > >> ~50 views, > > can't you really lower the views count? > It's impossible, :-( over 500'000 client use bind and we must use views to split load on another services. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
> В Пнд, 26/01/2009 в 16:16 -0800, JINMEI Tatuya / 神明達哉 пишет: > > http://www.jinmei.org/patch/bind9-lrucache.diff > > (should be cleanly applicable to 9.6). > > and let me know if it mitigates the problem? On 29.01.09 22:50, Dmitry Rybin wrote: > Oh, great work. I'll try tomorrow. > > Other recommendations: > > - I previously suggested using a separate cache-only view and forward > > all recursive queries to that view. Have you tried that? If you > > have, didn't it work as I hoped? > Yes, I try it. But I can't set ttl to 0. It didn't work. Recursive query > fails, and authoritative query back to clients with ttl 0 :( Yes, that is what "Setting TTL to 0" means. > ~50 views, can't you really lower the views count? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. - Holmes, what kind of school did you study to be a detective? - Elementary, Watson. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
В Пнд, 26/01/2009 в 16:16 -0800, JINMEI Tatuya / 神明達哉 пишет: > http://www.jinmei.org/patch/bind9-lrucache.diff > (should be cleanly applicable to 9.6). > and let me know if it mitigates the problem? > Oh, great work. I'll try tomorrow. > Other recommendations: > - I previously suggested using a separate cache-only view and forward > all recursive queries to that view. Have you tried that? If you > have, didn't it work as I hoped? Yes, I try it. But I can't set ttl to 0. It didn't work. Recursive query fails, and authoritative query back to clients with ttl 0 :( I increase memory on servers 2x QUAD CORE XEON up to 12Gb. PID USERNAME THR PRI NICE SIZERES STATE C TIME WCPU COMMAND 38634 bind11 40 3003M 2952M RUN2 159:28 46.44% named ~50 views, max-cache-size for most views 64M; bind uptime (after kernel: pid 667 (named), uid 53: exited on signal 11) - 2 days and 6 hours. built with '--localstatedir=/var' '--disable-linux-caps' '--with-randomdev=/dev/random' '--d isable-openssl-version-check' '--without-openssl' '--with-libxml2=/usr/local' '--without-idn' '--enable-largefile' '--enable-threads' '--prefix=/usr/local' ' --mandir=/usr/local/man' '--infodir=/usr/local/info/' '--build=x86_64-portbld-freebsd7.1' 'build_alias=x86_64-portbld-freebsd7.1' 'CC=cc' 'CFLAGS=-O2 -fno-st rict-aliasing -pipe' 'LDFLAGS= -rpath=/usr/lib:/usr/local/lib' 'CXX=c++' 'CXXFLAGS=-O2 -fno-strict-aliasing -pipe' rndc status: version: 9.6.0-P1 CPUs found: 8 worker threads: 8 On another server in same configuration bind works 2 days and die without core kernel: pid 682 (named), uid 53: exited on signal 11 Max memory per process - 12GB. May be FreeBSD x64 can't work more then X Gb per process? # cat /boot/loader.conf kern.maxdsiz="17179869184" # 16gb kern.dfldsiz="17179869184" # 16gb kern.maxssiz="134217728"# 128MB > - BIND 9.7 will have a new option "attach-cache" exactly for such an > extraordinary operational environment as yours: it allows multiple > views to share a single cache to save memory. I'll try to test 9.7 on one of the heavy load servers and post results to you. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
At Thu, 22 Jan 2009 09:12:11 +0300, Dmitry Rybin wrote: > > +50 views of zone data + memory for 10 clients + > > > > You have a 32bit build which will give a maximum of 2G data. > > > > You are just trying to cram too much into too small a place. > > OK. May be you can give any recomendation? As Mark said, having 50 views, each of which contains non-negligible amount of cache, is an excessive condition. Also, since the matching view is identified by linear search for every query, it may also impact your query processing performance. So, you'd primarily consider reducing the number of views anyway. Still, I noticed cache management may not work well (even with a single view) especially when it's multi-threaded and configured with a small max-cache-size such as 16MB. (It's ironical that using a small max-cache-size could hinder cache cleaning, resulting in larger memory footprints). I'm developing a fix to this problem. Can you try the patch available at: http://www.jinmei.org/patch/bind9-lrucache.diff (should be cleanly applicable to 9.6). and let me know if it mitigates the problem? Other recommendations: - I previously suggested using a separate cache-only view and forward all recursive queries to that view. Have you tried that? If you have, didn't it work as I hoped? - BIND 9.7 will have a new option "attach-cache" exactly for such an extraordinary operational environment as yours: it allows multiple views to share a single cache to save memory. --- JINMEI, Tatuya Internet Systems Consortium, Inc. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Actually thinking about your problem i just got an idea for a quick and
dirty solution that might just be it for you:
Keep running the views on your fontend nameserver but forward all
recursive queries to another recursive server via the "forward only;"
statement. IIRC that should cause BIND not to cache on the frontend
server.
Stefan
--
printk("CPU[%d]: Sending penguins to jail...",smp_processor_id());
linux-2.4.8/arch/sparc64/kernel/smp.c
___
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
On Wed, Jan 21, 2009 at 12:10:05PM +0300, Dmitry Rybin wrote:
> view "view0"{
> max-cache-size 16M;
> match-clients {
> XXX.XXX.XXX.XXX;
> };
> include "net-views/view0.conf";
> };
>
> [... skip 48 views ...]
>
> view "view50"{
> max-cache-size 8M;
> match-clients {
> XXX.XXX.XXX.XXX;
> };
> include "net-views/view50.conf";
> };
The way i read this you are using one view for each of the different
client IPs you have. Do you really need all of those or are you just
trying to have an internal and an external view for a range of clients?
The match-clients statement takes a list of IPs, CIDR ranges or ACLs.
Stefan
--
printk("%s: huh ? Who issued this format command ?\n")
linux-2.6.6/drivers/block/ps2esdi.c
___
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
> Matus UHLAR - fantomas wrote: > > This is _NOT_ a problem of BIND. This is a problem of its admin who can't > > read the docs and set up max-cache-size, which does exactly what is needed > > in this case. On 21.01.09 17:38, Dmitry Rybin wrote: > Hmm... And why bind allocate all system memory, if max-cache-size 16M? > And views... 50 views. 16*50=800M. Only 800M, this is not 3..4GB of > system memory. lower it down to e.g. 4-8MB to see if it helps a bit. But I'd think if 50 views is really needed here... and if you have 800 MB of cache and 4GB of used memory, I'd say that size of the cache is not the real problem btw is the max-cache-size really per-view? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. You have the right to remain silent. Anything you say will be misquoted, then used against you. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Mark Andrews wrote: >>> >> Hmm... And why bind allocate all system memory, if max-cache-size 16M? >> And views... 50 views. 16*50=800M. Only 800M, this is not 3..4GB of >> system memory. > > +50 views of zone data + memory for 10 clients + > > You have a 32bit build which will give a maximum of 2G data. > > You are just trying to cram too much into too small a place. OK. May be you can give any recomendation? file /usr/local/sbin/named /usr/local/sbin/named: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), for FreeBSD 7.1 (701100), dynamically linked (uses shared libs), FreeBSD-style, stripped ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
In message <49773369.4080...@corbina.net>, Dmitry Rybin writes: > Matus UHLAR - fantomas wrote: > > > > > This is _NOT_ a problem of BIND. This is a problem of its admin who can't > > read the docs and set up max-cache-size, which does exactly what is needed > > in this case. > > > > Hmm... And why bind allocate all system memory, if max-cache-size 16M? > And views... 50 views. 16*50=800M. Only 800M, this is not 3..4GB of > system memory. +50 views of zone data + memory for 10 clients + You have a 32bit build which will give a maximum of 2G data. You are just trying to cram too much into too small a place. Mark > ___ > bind-users mailing list > bind-users@lists.isc.org > https://lists.isc.org/mailman/listinfo/bind-users -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: mark_andr...@isc.org ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Matus UHLAR - fantomas wrote: > > This is _NOT_ a problem of BIND. This is a problem of its admin who can't > read the docs and set up max-cache-size, which does exactly what is needed > in this case. > Hmm... And why bind allocate all system memory, if max-cache-size 16M? And views... 50 views. 16*50=800M. Only 800M, this is not 3..4GB of system memory. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
> >>> On 20.01.09 12:49, Dmitry Rybin wrote: > >>>> How to disable cache in bind-9.6? ttl=0 - bad idea. > >> Matus UHLAR - fantomas wrote: > >>> if you know that setting TTL to 0 is a bad idea, why do yuo think that > >>> disabling a cache in BIND is not a bad idea? > > Dmitry Rybin wrote: > >> Because under high load cache grows to maximum system size and stop > >> responding to queues. This is known problem. > Alan Clegg wrote: > > This is NOT a "known problem" in 9.6. Please provide your configuration > > and logs that show the issue that you are having. On 21.01.09 12:10, Dmitry Rybin wrote: > this is known problem of all bind's. Bind grows up to 2Gb, become slowly > answer to queries and can't restart, only kill -9. FreeBSD 5.x7.1, > Linux 2.6. This is _NOT_ a problem of BIND. This is a problem of its admin who can't read the docs and set up max-cache-size, which does exactly what is needed in this case. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. It's now safe to throw off your computer. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Alan Clegg wrote:
> Dmitry Rybin wrote:
>> Matus UHLAR - fantomas wrote:
>>> On 20.01.09 12:49, Dmitry Rybin wrote:
>>>> How to disable cache in bind-9.6? ttl=0 - bad idea.
>>> if you know that setting TTL to 0 is a bad idea, why do yuo think that
>>> disabling a cache in BIND is not a bad idea?
>>>
>> Because under high load cache grows to maximum system size and stop
>> responding to queues. This is known problem.
>
> This is NOT a "known problem" in 9.6. Please provide your configuration
> and logs that show the issue that you are having.
>
> AlanC
>
this is known problem of all bind's. Bind grows up to 2Gb, become slowly
answer to queries and can't restart, only kill -9. FreeBSD 5.x7.1,
Linux 2.6.
options {
directory "/etc/namedb";
max-cache-size 16M;
max-cache-ttl 3600;
max-ncache-ttl 1800;
cleaning-interval 10;
transfers-in 1000;
transfers-out 1000;
transfers-per-ns 100;
minimal-responses yes;
allow-recursion {
xxx.xxx.xxx.xxx;
};
recursive-clients 1;
clients-per-query 80;
max-clients-per-query 100 ;
view "world" {
zone "." {
type hint;
file "named.root";
};
zone "0.0.127.IN-ADDR.ARPA" {
type master;
file "localhost.rev";
};
view "view0"{
max-cache-size 16M;
match-clients {
XXX.XXX.XXX.XXX;
};
include "net-views/view0.conf";
};
[... skip 48 views ...]
view "view50"{
max-cache-size 8M;
match-clients {
XXX.XXX.XXX.XXX;
};
include "net-views/view50.conf";
};
___
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
> On Jan 20, 2009, at 4:49 AM, Dmitry Rybin wrote: > >How to disable cache in bind-9.6? ttl=0 - bad idea. On 20.01.09 14:44, John Wobus wrote: > Disabling the cache makes sense if the purpose of your > nameserver is to provide your authoritative zone data and you > have a different nameserver to handle your site's general > DNS queries. in such case it's much better to disable recursion and not use such server for resolution, unless it's a MUST (e.g. firewalls). -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Silvester Stallone: Father of the RISC concept. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Disabling the cache makes sense if the purpose of your nameserver is to provide your authoritative zone data and you have a different nameserver to handle your site's general DNS queries. TTL settings are part of authoritative zone data, which is completely independent of whether you disable caching in the nameserver. On Jan 20, 2009, at 4:49 AM, Dmitry Rybin wrote: Hello! How to disable cache in bind-9.6? ttl=0 - bad idea. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Dmitry Rybin wrote: > Matus UHLAR - fantomas wrote: >> On 20.01.09 12:49, Dmitry Rybin wrote: >>> How to disable cache in bind-9.6? ttl=0 - bad idea. >> if you know that setting TTL to 0 is a bad idea, why do yuo think that >> disabling a cache in BIND is not a bad idea? >> > > Because under high load cache grows to maximum system size and stop > responding to queues. This is known problem. This is NOT a "known problem" in 9.6. Please provide your configuration and logs that show the issue that you are having. AlanC signature.asc Description: OpenPGP digital signature ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
> > On 20.01.09 12:49, Dmitry Rybin wrote: > >> How to disable cache in bind-9.6? ttl=0 - bad idea. > Matus UHLAR - fantomas wrote: > > if you know that setting TTL to 0 is a bad idea, why do yuo think that > > disabling a cache in BIND is not a bad idea? On 20.01.09 18:39, Dmitry Rybin wrote: > Because under high load cache grows to maximum system size and stop > responding to queues. This is known problem. Did you set up maximum cache size to a sane value? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Enter any 12-digit prime number to continue. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Disable cache in bind 9.6
Matus UHLAR - fantomas wrote: > On 20.01.09 12:49, Dmitry Rybin wrote: >> How to disable cache in bind-9.6? ttl=0 - bad idea. > > if you know that setting TTL to 0 is a bad idea, why do yuo think that > disabling a cache in BIND is not a bad idea? > Because under high load cache grows to maximum system size and stop responding to queues. This is known problem. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Disable cache in bind 9.6
Hello! How to disable cache in bind-9.6? ttl=0 - bad idea. ___ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
