Re: response-rate-limiting - "window" explained?

[Tom]

On 01/09/2018 05:11 PM, Tony Finch wrote:

Tom  wrote:


Slip is set to "0" (always drop). After stopping the flood, I'm immediately
able to query the same record (www.example.com) with a positive answer. Does
the "window 5;" or "window 30;" or "window 3600;" possibly has no effect?


The script below works for me. My server is configured with

responses-per-second 2;
slip 0;

The script "floods" at 10 qps for 15 seconds (enough to fill the window),
then drops to 1qps and times how long it takes to recover. At the end it
says the recovery time was 20 seconds. This is a bit more than the window
size because I wasn't completely quiet during the recovery time.



Many thanks for this hint/script. The rate-limit works as explained. My 
misunderstanding was, that I didn't set the query-process(es) in the 
background.

Kind regards,
Tom




#!/bin/sh

set -eu

dig='dig +ignore +tries=1 +timeout=1 +norec dotat.at @grey.dotat.at'

start=$(date +%s)

while [ $(date +%s) -lt $(($start + 15)) ]
do
$dig | grep aa &
sleep 0.1
done

end=$(date +%s)

while ! $dig | grep aa
do
:;
done

echo $(( $(date +%s) - $end ))



Tony.


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tony Finch]

Bob Harold  wrote:

>That's a good test, with the default window of 15 seconds, but could you
> please repeat it with a window like 120, and see if it changes
> accordingly?

With a window of 5 I got a recovery time of 6s.

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Trafalgar: Variable 3 for a time in southeast, otherwise south 6 to gale 8,
veering northwest 5 or 6 later. Moderate or rough, becoming very rough later
in northwest. Rain or showers. Good, occasionally poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Bob Harold]

On Tue, Jan 9, 2018 at 11:11 AM, Tony Finch  wrote:

> Tom  wrote:
> >
> > Slip is set to "0" (always drop). After stopping the flood, I'm
> immediately
> > able to query the same record (www.example.com) with a positive answer.
> Does
> > the "window 5;" or "window 30;" or "window 3600;" possibly has no effect?
>
> The script below works for me. My server is configured with
>
> responses-per-second 2;
> slip 0;
>
> The script "floods" at 10 qps for 15 seconds (enough to fill the window),
> then drops to 1qps and times how long it takes to recover. At the end it
> says the recovery time was 20 seconds. This is a bit more than the window
> size because I wasn't completely quiet during the recovery time.
>
> 
>
> #!/bin/sh
>
> set -eu
>
> dig='dig +ignore +tries=1 +timeout=1 +norec dotat.at @grey.dotat.at'
>
> start=$(date +%s)
>
> while [ $(date +%s) -lt $(($start + 15)) ]
> do
> $dig | grep aa &
> sleep 0.1
> done
>
> end=$(date +%s)
>
> while ! $dig | grep aa
> do
> :;
> done
>
> echo $(( $(date +%s) - $end ))
>
> 
>
> Tony.
> --
> f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h
> punycode
> Irish Sea: Southeast 5 to 7, becoming cyclonic 4 or 5 later. Moderate or
> rough, becoming slight or moderate. Occasional rain. Good, occasionally
> poor.
>

Tony,
   That's a good test, with the default window of 15 seconds, but could you
please repeat it with a window like 120, and see if it changes
accordingly?  Tom's testing indicates that the 'window' parameter has no
effect, which seems wrong.

-- 
Bob Harold
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tony Finch]

Tom  wrote:
>
> Slip is set to "0" (always drop). After stopping the flood, I'm immediately
> able to query the same record (www.example.com) with a positive answer. Does
> the "window 5;" or "window 30;" or "window 3600;" possibly has no effect?

The script below works for me. My server is configured with

responses-per-second 2;
slip 0;

The script "floods" at 10 qps for 15 seconds (enough to fill the window),
then drops to 1qps and times how long it takes to recover. At the end it
says the recovery time was 20 seconds. This is a bit more than the window
size because I wasn't completely quiet during the recovery time.



#!/bin/sh

set -eu

dig='dig +ignore +tries=1 +timeout=1 +norec dotat.at @grey.dotat.at'

start=$(date +%s)

while [ $(date +%s) -lt $(($start + 15)) ]
do
$dig | grep aa &
sleep 0.1
done

end=$(date +%s)

while ! $dig | grep aa
do
:;
done

echo $(( $(date +%s) - $end ))



Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Irish Sea: Southeast 5 to 7, becoming cyclonic 4 or 5 later. Moderate or
rough, becoming slight or moderate. Occasional rain. Good, occasionally poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tom]

On 01/09/2018 02:49 PM, Tony Finch wrote:

Tom  wrote:


If I set the "responses-per-second 5;" and the "window 30;", then begin
flooding (the responses are correctly dropped), then stop flooding, then
querying the nameserver from the same source for the same RR, I'll get
immediately the right answer.

Any explanations for this behavior?


Try more than once - you are probably seeing the effect of the "slip"
setting, which is supposed to allow legitimate clients to get answers even
when they are being spoofed by a DDoS attack.


I tried many times with different values for "window" (window 5;, window 
30;, window 3600;) Always the same effect with the following command:
while true; do echo -n "$(date)   "; dig +short +ignore +tries=1 
@x.x.x.x www.example.com; sleep .01; done


Slip is set to "0" (always drop). After stopping the flood, I'm 
immediately able to query the same record (www.example.com) with a 
positive answer. Does the "window 5;" or "window 30;" or "window 3600;" 
possibly has no effect?


Thank you.
Kind regards,
Tom



Also, if you are using DiG then to see the proper effect you'll want to
set the +ignore +tries=1 options (and maybe +timeout=1).

Tony.


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tony Finch]

Tom  wrote:
>
> If I set the "responses-per-second 5;" and the "window 30;", then begin
> flooding (the responses are correctly dropped), then stop flooding, then
> querying the nameserver from the same source for the same RR, I'll get
> immediately the right answer.
>
> Any explanations for this behavior?

Try more than once - you are probably seeing the effect of the "slip"
setting, which is supposed to allow legitimate clients to get answers even
when they are being spoofed by a DDoS attack.

Also, if you are using DiG then to see the proper effect you'll want to
set the +ignore +tries=1 options (and maybe +timeout=1).

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Fair Isle, Faeroes, Southeast Iceland: Southeasterly 6 to gale 8, occasionally
severe gale 9, except in Faeroes. Rough or very rough, occasionally high in
Faeroes and Southeast Iceland. Occasional rain. Good, occasionally poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tony Finch]

Tom  wrote:

> Mmmh...I can't verify the meaning of the "window"-value. In my flood-tests, it
> makes no differences, if I set this value to 5 or 60 or even 3600.

You'll only notice the window if you pause your flood test - it's
basically the recovery time. (This is why the "stop limiting" log message
is misleading: RRL actually stops limiting after the window passes, but it
logs 60s later.)

The relevant part of the ARM is:

> Response rate limiting uses a "credit" or "token bucket" scheme.  Each
> combination of identical response and client has a conceptual account
> that earns a specified number of credits every second. A prospective
> response debits its account by one. Responses are dropped or truncated
> while the account is negative.  Responses are tracked within a rolling
> window of time which defaults to 15 seconds, but can be configured with
> the window option to any value from 1 to 3600 seconds (1 hour).  The
> account cannot become more positive than the per-second limit or more
> negative than window times the per-second limit.  When the specified
> number of credits for a class of responses is set to 0, those responses
> are not rate limited.

So, during your flood test, your client's token bucket will have its
minimym value, -1 * window * responses-per-second. When it stops, the
token bucket value will increase by `responses-per-second` each second, so
it will increase past zero after `window` seconds, and at that point RRL
will stop dropping responses.

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Rockall: South 6 to gale 8, backing southeast 7 to severe gale 9, perhaps
storm 10 later. Very rough or high. Occasional rain. Good, occasionally poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tom]

Mmmh...I can't verify the meaning of the "window"-value. In my 
flood-tests, it makes no differences, if I set this value to 5 or 60 or 
even 3600. The only value which affects the behavior is "slip" and 
"responses-per-second".
When I fast-querying (100r/s) the nameserver with the following 
configuration:.

rate-limit {
responses-per-second 5;
slip 0;
window 5;
log-only no;
};


the nameserver answers always every five seconds for five requests:
Mon Jan  8 07:36:05 CET 2018   querying example.org
Mon Jan  8 07:36:05 CET 2018   querying example.org
Mon Jan  8 07:36:05 CET 2018   querying example.org
Mon Jan  8 07:36:05 CET 2018   querying example.org
Mon Jan  8 07:36:05 CET 2018   querying example.org
Mon Jan  8 07:36:10 CET 2018   querying example.org
Mon Jan  8 07:36:10 CET 2018   querying example.org
Mon Jan  8 07:36:10 CET 2018   querying example.org
Mon Jan  8 07:36:10 CET 2018   querying example.org
Mon Jan  8 07:36:10 CET 2018   querying example.org
Mon Jan  8 07:36:15 CET 2018   querying example.org
Mon Jan  8 07:36:15 CET 2018   querying example.org
Mon Jan  8 07:36:15 CET 2018   querying example.org
Mon Jan  8 07:36:15 CET 2018   querying example.org
Mon Jan  8 07:36:15 CET 2018   querying example.org

And here it makes no differences, if I set the "window"-value to 5 or 60 
or 3600.



Any hints / explanation for the behavior of the "window"-value?


Many thanks.
Tom


On 01/05/2018 07:27 PM, Tony Finch wrote:

Tom  wrote:


Could someone explain the problem here? Why do I never have to wait longer
than about 5s until I'm able to query the nameserver from the unique client
with the same query again?


The 60s is the delay after a client has stopped making queries when the
information about that client can be forgotten. RRL logs "stop limiting"
at that point which is a bit misleading.

Tony.


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tony Finch]

Tom  wrote:

> Could someone explain the problem here? Why do I never have to wait longer
> than about 5s until I'm able to query the nameserver from the unique client
> with the same query again?

The 60s is the delay after a client has stopped making queries when the
information about that client can be forgotten. RRL logs "stop limiting"
at that point which is a bit misleading.

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Northwest Fitzroy, Sole: Northwesterly, veering northerly or northeasterly, 6
to gale 8. Very rough occasionally high. Thundery showers. Good occasionally
poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tom]

I've tested several "window"-values (5-3600) with fast-querying the 
nameserver from one single client with always the same query. As 
explained by Tony the "window" means the time, which the client must 
wait, after he stops fast-querying the nameserver while he was 
successfully dropped.


In my tests, I never had to wait for about more than about 5s.
I've configured rate-limits like this:
   rate-limit {
responses-per-second 5;
slip 0;
window 5;
log-only no; };

Could someone explain the problem here? Why do I never have to wait 
longer than about 5s until I'm able to query the nameserver from the 
unique client with the same query again?


Many thanks.
Kind regards,
Tom



On 03/27/2017 11:33 AM, Tony Finch wrote:

Tom  wrote:


Can someone explain the behaviour of "window" in the rate-limit-context?


It basically determines the time after a client that was querying very
fast but then stopped is allowed to receive responses again.

When a client repeats a query, its counter is decremented until it reaches
the minimum `-1 * window * responses-per-second`. Its counter is
incremented by `responses-per-second` each second, so after the client
stops querying it will be `window` seconds before the counter becomes
positive which means the client is allowed to receive responses again.

Tony.


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: response-rate-limiting - "window" explained?

[Tony Finch]

Tom  wrote:

> Can someone explain the behaviour of "window" in the rate-limit-context?

It basically determines the time after a client that was querying very
fast but then stopped is allowed to receive responses again.

When a client repeats a query, its counter is decremented until it reaches
the minimum `-1 * window * responses-per-second`. Its counter is
incremented by `responses-per-second` each second, so after the client
stops querying it will be `window` seconds before the counter becomes
positive which means the client is allowed to receive responses again.

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Irish Sea: East 4 or 5, decreasing 3 at times. Mainly slight. Fair. Good.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

response-rate-limiting - "window" explained?

[Tom]

Hi
Can someone explain the behaviour of "window" in the rate-limit-context? 
I've tried "responses-per-second 10; window 3;" and had the same results 
as "responses-per-second 10; window 5;".


Any simple explanation for the "window"-directive?

Many thanks.
Kind regards,
Tom
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users