Re: big improvement in BIND9 auth-server startup time

[Doug Barton]

On 08/09/2011 11:17, J. Thomsen wrote:
> On Tue, 9 Aug 2011 17:18:13 +,Evan Hunt  wrote:
> 
>>> The master now loaded about 650 zones/sec.
>>> The slave did not change from the usual 120 zones/sec.
>>
>> I'm purely guessing, but I wonder if there's some rate-limiting
>> due to the SOA queries slaves have to send to their masters.
>>
> 
> You are probably right.
> 
> Increasing 
> 
> transfers-in/out 
> transfers-per-ns
> serial-query-rate
> 
> on both master and slave improved the slave load time considerably to 577 
> zones/sec.
> Which one is the important one to increase, I don't know.

Assuming that the zones were mostly up to date, it was serial-query-rate.

Also, FYI, you're getting pretty close to the point where the
master/slave configuration is going to become a bottleneck. If you plan
to grow much beyond this point you should start looking at making all of
your systems masters and using an out of band method to update the zones.


hth,

Doug

-- 

Nothin' ever doesn't change, but nothin' changes much.
-- OK Go

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price.  :)  http://SupersetSolutions.com/

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: big improvement in BIND9 auth-server startup time

[J. Thomsen]

On Tue, 9 Aug 2011 17:18:13 +,Evan Hunt  wrote:

>> The master now loaded about 650 zones/sec.
>> The slave did not change from the usual 120 zones/sec.
>
>I'm purely guessing, but I wonder if there's some rate-limiting
>due to the SOA queries slaves have to send to their masters.
>

You are probably right.

Increasing 

transfers-in/out 
transfers-per-ns
serial-query-rate

on both master and slave improved the slave load time considerably to 577 
zones/sec.
Which one is the important one to increase, I don't know.


- Jørgen Thomsen
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: big improvement in BIND9 auth-server startup time

[Evan Hunt]

> The master now loaded about 650 zones/sec.
> The slave did not change from the usual 120 zones/sec.

Interesting, thanks for bringing that to my attention (it hadn't
occurred to me to test with a mostly-slave system).

I'm purely guessing, but I wonder if there's some rate-limiting
due to the SOA queries slaves have to send to their masters.

I very much doubt the raw zonefile format is the problem. 
Generally that'll reduce the zone loading time by almost half.

-- 
Evan Hunt -- e...@isc.org
Internet Systems Consortium, Inc.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: big improvement in BIND9 auth-server startup time

[J. Thomsen]

On Wed, 03 Aug 2011 15:45:25 -0500,Barry Finkel  wrote:

>
>I did not see any improvement in start-up time. 


Neither did I at my first test on a primarily slave DNS with raw format 
zonefiles

Next test was on a master and slave with 60K small different zones.

The master now loaded about 650 zones/sec.
The slave did not change from the usual 120 zones/sec.

Difference ?
Task pool 599.
Hardware should be the same.

The master is loading text format zone files.
The slave is loading raw format zone files.

Somebody should check, if there is something in raw format files, which is 
slowing things
down. CPU and I/O seemed to be used very moderately.


- Jørgen Thomsen
 
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: big improvement in BIND9 auth-server startup time

[Mark K. Pettit]

Not sure where to report this, but there's a problem in the documentation of 
BIND 9.7.4, as distributed by ISC.

The Release Notes included in the bind-9.7.4 tarball, as well as the release 
notes on the web site:

  ftp://ftp.isc.org/isc/bind9/9.7.4/RELEASE-NOTES-BIND-9.7.4.html

state that the environment variable to improve loading times for name servers 
with large numbers of zones is:

  BIND9_ZONE_TASKS_HINTS

But that is incorrect.  The actual variable name is:

  BIND9_ZONE_TASKS_HINT

I discovered this because I was curious how this change was implemented (I had 
not seen the blog post 
http://www.isc.org/community/blog/201107/major-improvement-bind-9-startup-performance
 when I started to look into this), and couldn't find the string 
"BIND9_ZONE_TASKS_HINTS" anywhere in the source code.

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: big improvement in BIND9 auth-server startup time

[Barry Finkel]

Evan Hunt wrote last July 13:


-- People who operate big authoritative name servers (particularly with
large numbers of small zones, e.g., for domain hosting and parking),
and have had trouble with slow startup, may find this information
useful:

http://www.isc.org/community/blog/201107/major-improvement-bind-9-startup-performance


I ran some tests yesterday on an Ubuntu Hardy virtual machine.
I was using 9.7.3-P3 and 9.7.4.  There is nothing running
on the box except BIND and the Hardy operating system.

The DNS configuration file has a handful of test zones.
It also has our list of 202,818 spyware/malware zones,
each using the same zone file, which has one NS record and

 @   IN  A  a.b.c.d
 *   IN  A  a.b.c.d

The shutdown time for 9.7.3-P3 was 1:27:11 (1.5 hours).
The timing was from the "./rndc stop" until the "exiting" message.
I did three shutdowns of 9.7.4 with the default

 BIND9_ZONE_TASKS_HINT=8

value.  The shutdown times were 7:17, 7:07, and 7:35.
The shutdown time improved DRAMATICALLY.

I did three start-ups of 9.7.4:

  BZTH   Time
---  
default  4:57
default  5:04
  1020   5:19

I did not see any improvement in start-up time.  The
last start-up did produce the message

 Using 1020 tasks for zone loading

The first two start-ups produced the message

 Using 101 tasks for zone loading

The CHANGES file says that the default is 8.

I need to run some more tests.
--
Barry S. Finkel
Computing and Information Systems Division
Argonne National Laboratory  Phone:+1 (630) 252-7277
9700 South Cass Avenue   Facsimile:+1 (630) 252-4601
Building 240, Room 5.B.8 Internet: bsfin...@anl.gov
Argonne, IL   60439-4828 IBMMAIL:  I1004994
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: big improvement in BIND9 auth-server startup time

[Matthew Pounsett]

On 2011/07/13, at 11:15, Evan Hunt wrote:

> 
> People who operate big authoritative name servers (particularly with
> large numbers of small zones, e.g., for domain hosting and parking),
> and have had trouble with slow startup, may find this information
> useful:
> 
> http://www.isc.org/community/blog/201107/major-improvement-bind-9-startup-performance

Thanks Evan.  I'll also point out to people this PDF which has some more detail 
on the actual performance improvements: 


Was there any work done on measuring the impact on 'reconfig' times, or is that 
an entirely separate piece of code?  I'm curious if the change makes any 
difference to adding large numbers of zones to an already-running server.  e.g. 
if I've got a server handling 500,000 zones and add another 20k, 50k, or 100k 
via 'reconfig'.

Thanks to the ISC folk for doing that work!
   Matt


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: big improvement in BIND9 auth-server startup time

[Jan-Piet Mens]

Evan,

> may find this information useful:

very useful and quite impressive.

-JP
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users