> Another disconcerting fact is that the cache hit rate is 54% when its
> still OK but drops to 26% when you get trouble. Take a look at that
> dump-requestlist, and also perform a unbound-control stats_noreset >
> snapshot and look at it: the number of unwanted replies in particular.
> Certain forms of Kaminsky-eeks! would tend to blast lots of different
> queries to a resolver...
We are attempting to use Unbound as a DNS resolver/cache for a spider.
In a typical hour we are looking up several million domains but
typically only need to resolve it a handful of times per day.
We boosted outgoing-range to 8192, reduced jostle-timeout to 200, and
eliminated the min-ttl.
We also tried boosting the slabs counters to various values between 4 and 512.
None of these made any noticeable difference.
CPU usage becomes very low when exceeded begins to climb. Does unbound
sleep or spin while waiting for a lock?
Additional thoughts? We are going to attempt using much much smaller
cache values. I tend to believe it is the cache management from
dealing with several million different domains, but of course I have
no proof of that.
>> verbosity: 1
>> statistics-interval: 300
>> statistics-cumulative: yes
>> extended-statistics: no
>> num-threads: 4
>> outgoing-range: 2048
>> msg-cache-size: 3G
>> msg-cache-slabs: 8
>> num-queries-per-thread: 2048
>> jostle-timeout: 1000
>> so-rcvbuf: 8M
>> rrset-cache-size: 6G
>> rrset-cache-slabs: 8
>> cache-min-ttl: 7776000
>> harden-glue: yes
>> harden-dnssec-stripped: yes
>> harden-referral-path: yes
>> use-caps-for-id: yes
>> unwanted-reply-threshold: 1000
>> val-clean-additional: yes
>> val-permissive-mode: no
>> key-cache-slabs: 8
>> neg-cache-size: 1g
> On 01/19/2010 11:15 PM, Rod Taylor wrote:
>> We are trying unbound out. Machine is quad CPU with 16GB ram. I'm
>> picking on thread 2 because it shows the issue best as it seems to
>> receive up to 6 times as many queries as the other 3 threads. As you
>> can see, the time to execute climbs gradually through the period and
>> starts to "exceed" allowed limits about mid-way down the stats results
>> shown here.
>>
>> Config bits.
>> server:
>> verbosity: 1
>> statistics-interval: 300
>> statistics-cumulative: yes
>> extended-statistics: no
>> num-threads: 4
>> outgoing-range: 2048
>> msg-cache-size: 3G
>> msg-cache-slabs: 8
>> num-queries-per-thread: 2048
>> jostle-timeout: 1000
>> so-rcvbuf: 8M
>> rrset-cache-size: 6G
>> rrset-cache-slabs: 8
>> cache-min-ttl: 7776000
>> harden-glue: yes
>> harden-dnssec-stripped: yes
>> harden-referral-path: yes
>> use-caps-for-id: yes
>> unwanted-reply-threshold: 1000
>> val-clean-additional: yes
>> val-permissive-mode: no
>> key-cache-slabs: 8
>> neg-cache-size: 1g
>>
>>
>> Below are snippets from the statistics taken roughly every 5 minutes
>> from startup over a period of 7 hours.
>>
>> server stats for thread 2: 420 queries, 284 answers from cache, 136
>> recursions
>> server stats for thread 2: requestlist max 21 avg 16.2647 exceeded 0
>> server stats for thread 2: 3189 queries, 3053 answers from cache, 136
>> recursions
>> server stats for thread 2: requestlist max 21 avg 16.2647 exceeded 0
>> server stats for thread 2: 3375 queries, 3239 answers from cache, 136
>> recursions
>> server stats for thread 2: requestlist max 21 avg 16.2647 exceeded 0
>> server stats for thread 2: 4560 queries, 4424 answers from cache, 136
>> recursions
>> server stats for thread 2: requestlist max 21 avg 16.2647 exceeded 0
>> server stats for thread 2: 32129 queries, 13211 answers from cache,
>> 18918 recursions
>> server stats for thread 2: requestlist max 1196 avg 620.017 exceeded 0
>> server stats for thread 2: 227635 queries, 63600 answers from cache,
>> 164035 recursions
>> server stats for thread 2: requestlist max 1196 avg 605.375 exceeded 0
>> server stats for thread 2: 412374 queries, 113701 answers from cache,
>> 298673 recursions
>> server stats for thread 2: requestlist max 1196 avg 649.591 exceeded 0
>> server stats for thread 2: 591627 queries, 164560 answers from cache,
>> 427067 recursions
>> server stats for thread 2: requestlist max 1196 avg 685.99 exceeded 0
>> server stats for thread 2: 762418 queries, 214876 answers from cache,
>> 547542 recursions
>> server stats for thread 2: requestlist max 1196 avg 725.279 exceeded 0
>> server stats for thread 2: 925072 queries, 265281 answers from cache,
>> 659791 recursions
>> server stats for thread 2: requestlist max 1196 avg 766.182 exceeded 0
>> server stats for thread 2: 1082217 queries, 315509 answers from cache,
>> 766708 recursions
>> server stats for thread 2: requestlist max 1213 avg 803.033 exceeded 0
>> server stats for thread 2: 1240514 queries,
