Re: Request for review of performance advice

[Browne, Stuart via bind-users]

Just one quick one before I run off to lunch with regards to section 2:

- Try to avoid crossing NUMA boundaries. At high throughput, the context 
switching and far memory calls kills performance.

Stuart

From: bind-users  on behalf of Victoria Risk 

Date: Wednesday, 8 July 2020 at 11:58
To: bind-users 
Subject: Request for review of performance advice

A while ago we created a KB article with tips on how to improve your 
performance with our Kea dhcp server. The tips were fairly obvious to our 
developers and this was pretty successful. We would like to do something 
similar for BIND, provide a dozen or so tips for how to maximize your 
throughput with BIND. However, as usual, everything is more complicated with 
BIND.

Can those of you who care about performance, who have worked to improve your 
performance, share some of your suggestions that have the most impact?  Please 
also comment if you think any of these ideas below are stupid or dangerous. I 
have combined advice for resolvers and for authoritative servers, I hope it is 
clear which is which...

The ideas we have fall into four general categories:

System design
1a) Use a load balancer to specialize your resolvers and maximize your cache 
hit ratio.  A load balancer is traditionally designed to spread the traffic out 
evenly among a pool of servers, but it can also be used to concentrate related 
queries on one server to make its cache as hot as possible. For example, if all 
queries for domains in .info are sent to one server in a pool, there is a 
better chance that an answer will be in the cache there.

1b) If you have a large authoritative system with many servers, consider 
dedicating some machines to propagate transfers. These machines, called 
transfer servers, would not answer client queries, but just send notifies and 
process IXFR requests.


1c) Deploy ghost secondaries.  If you store copies of authoritative zones on 
resolvers (resolvers as undelegated secondaries), you can avoid querying those 
authoritative zones. The most obvious uses of this would be mirroring the root 
zone locally or mirroring your own authoritative zones on your resolver.

we have other system design ideas that we suspect would help, but we are not 
sure, so I will wait to see if anyone suggests them.

OS settings and the system environment
2a) Run on bare metal if possible, not on virtual machines or in the cloud. 
(any idea how much difference this makes? the only reference we can cite is 
pretty out of date - 
https://urldefense.com/v3/__https:/indico.dns-oarc.net/event/19/contributions/234/attachments/217/411/DNS_perf_OARC_Apr_14.pdf__;!!N14HnBHF!rk-RfzR0chw8mToGMWAwQAF_WiiXKZM3KXol3WR8YPytPoI_cWyNe5BZ_rsEqdV7T9SIQ1M$
 )

2b) Consider using with-tuning-large. 
(https://urldefense.com/v3/__https:/kb.isc.org/docs/aa-01314__;!!N14HnBHF!rk-RfzR0chw8mToGMWAwQAF_WiiXKZM3KXol3WR8YPytPoI_cWyNe5BZ_rsEqdV7ufSMbnU$)
 This is a compile time option, so not something you can switch on and off 
during production. 

2c) Consider which R/W lock choice you want to use - 
https://urldefense.com/v3/__https:/kb.isc.org/docs/choosing-a-read-write-lock-implementation-to-use-with-named__;!!N14HnBHF!rk-RfzR0chw8mToGMWAwQAF_WiiXKZM3KXol3WR8YPytPoI_cWyNe5BZ_rsEqdV7mVVUg4A$
 For the highest tested query rates (> 100,000 queries per second), pthreads 
read-write locks with hyper-threading enabled seem to be the best-performing 
choice by far.


2d) Pay attention to your choice of NIC cards. We have found wide variations in 
their performance. (Can anyone suggest what specifically to look for?)


2e) Make sure your socket send buffers are big enough. (not sure if this is 
obsolete advice, do we need to tell people how to tell if their buffers are 
causing delays?)

2f) When the number of CPUs is very large (32 or more), the increase in UDP 
listeners may not provide any performance improvement and might actually reduce 
throughput slightly due to the overhead of the additional structures and tasks. 
We suggest trying different values of -U to find the optimal one for your 
production environment.




named Features
3a) Minimize logging. Query logging is expensive (can cost you 20% or more of 
your throughput) so don’t do it unless you are using the logs for something. 
Logging with dnstap is lower impact, but still fairly expensive. Don’t run in 
debug mode unless necessary. 


3b) Use named.conf option minimal-responses yes; to reduce the amount of work 
that named needs to do to assemble the query response as well as reducing the 
amount of outbound traffic


3c) Disable synth-from-dnssec. While this seemed like a good idea, it turns 
out, in practice it does not improve performance.


3d) Tune your zone transfers. 
(https://urldefense.com/v3/__https:/kb.isc.org/docs/aa-00726__;!!N14HnBHF!rk-RfzR0chw8mToGMWAwQAF_WiiXKZM3KXol3WR8YPytPoI_cWyNe5BZ_rsEqdV7K_7-VnQ$)
When tuning the behavior of the primary, there are several factors that you can 
control:
- The rate of notifications of 

Request for review of performance advice

[Victoria Risk]

A while ago we created a KB article with tips on how to improve your 
performance with our Kea dhcp server. The tips were fairly obvious to our 
developers and this was pretty successful. We would like to do something 
similar for BIND, provide a dozen or so tips for how to maximize your 
throughput with BIND. However, as usual, everything is more complicated with 
BIND.

Can those of you who care about performance, who have worked to improve your 
performance, share some of your suggestions that have the most impact?  Please 
also comment if you think any of these ideas below are stupid or dangerous. I 
have combined advice for resolvers and for authoritative servers, I hope it is 
clear which is which...

The ideas we have fall into four general categories:

System design
1a) Use a load balancer to specialize your resolvers and maximize your cache 
hit ratio.  A load balancer is traditionally designed to spread the traffic out 
evenly among a pool of servers, but it can also be used to concentrate related 
queries on one server to make its cache as hot as possible. For example, if all 
queries for domains in .info are sent to one server in a pool, there is a 
better chance that an answer will be in the cache there.

1b) If you have a large authoritative system with many servers, consider 
dedicating some machines to propagate transfers. These machines, called 
transfer servers, would not answer client queries, but just send notifies and 
process IXFR requests.

1c) Deploy ghost secondaries.  If you store copies of authoritative zones on 
resolvers (resolvers as undelegated secondaries), you can avoid querying those 
authoritative zones. The most obvious uses of this would be mirroring the root 
zone locally or mirroring your own authoritative zones on your resolver.

we have other system design ideas that we suspect would help, but we are not 
sure, so I will wait to see if anyone suggests them.

OS settings and the system environment
2a) Run on bare metal if possible, not on virtual machines or in the cloud. 
(any idea how much difference this makes? the only reference we can cite is 
pretty out of date - 
https://indico.dns-oarc.net/event/19/contributions/234/attachments/217/411/DNS_perf_OARC_Apr_14.pdf
 

 )

2b) Consider using with-tuning-large. (https://kb.isc.org/docs/aa-01314 
) This is a compile time option, so not 
something you can switch on and off during production. 

2c) Consider which R/W lock choice you want to use - 
https://kb.isc.org/docs/choosing-a-read-write-lock-implementation-to-use-with-named
 

 For the highest tested query rates (> 100,000 queries per second), pthreads 
read-write locks with hyper-threading enabled seem to be the best-performing 
choice by far.

2d) Pay attention to your choice of NIC cards. We have found wide variations in 
their performance. (Can anyone suggest what specifically to look for?)

2e) Make sure your socket send buffers are big enough. (not sure if this is 
obsolete advice, do we need to tell people how to tell if their buffers are 
causing delays?)

2f) When the number of CPUs is very large (32 or more), the increase in UDP 
listeners may not provide any performance improvement and might actually reduce 
throughput slightly due to the overhead of the additional structures and tasks. 
We suggest trying different values of -U to find the optimal one for your 
production environment.


named Features
3a) Minimize logging. Query logging is expensive (can cost you 20% or more of 
your throughput) so don’t do it unless you are using the logs for something. 
Logging with dnstap is lower impact, but still fairly expensive.  Don’t run in 
debug mode unless necessary. 

3b) Use named.conf option minimal-responses yes; to reduce the amount of work 
that named needs to do to assemble the query response as well as reducing the 
amount of outbound traffic

3c) Disable synth-from-dnssec. While this seemed like a good idea, it turns 
out, in practice it does not improve performance.

3d) Tune your zone transfers.  (https://kb.isc.org/docs/aa-00726 
)
When tuning the behavior of the primary, there are several factors that you can 
control:

- The rate of notifications of changes to secondary servers (serial-query-rate 
and notify-delay)

- Limits on concurrent zone transfers (transfers-out, tcp-clients, 
tcp-listen-queue, reserved-sockets)

- Efficiency/management options (max-transfer-time-out, max-transfer-idle-out, 
transfer-format)

The most important options to focus on are transfers-out, serial-query-rate, 
tcp-clients and tcp-listen-queue.

4e) If you use RPZ, consider using qnane-wait-recurse. We have had issues with 
RPZ transfers impacting query performance in resolvers. In general, more 
smaller RPZ 

Re: rndc valid key types

[Evan Hunt]

On Tue, Jul 07, 2020 at 04:32:37PM -0700, Gregory Sloop wrote:
> I've seen reports that only HMAC-MD5 is the only valid key type.

That was the case at one time, but hasn't been for years.

> Is there any (security) reason/implications to use something "better"
> than MD5?

MD5 is broken (as is SHA1). In this specific context, a forged rndc message
is probably impracticable on any reasonable time scale, and I wouldn't fear
for security if I were using them.  *But*, they're broken, and crypto
people don't like keeping broken things around, so I wouldn't count on them
being supported forever. We've already removed MD5 support in the context
of DNSSEC keys; TSIG could come next.

So, if you want to generate a key and not have to worry about generating
another one in a year or two, I would advise against MD5 or SHA1.

> Is there any reason not to select the strongest - HMAC-SHA512?

No, go ahead. I tend to use sha256, just because it's the default
from rndc-confgen.

-- 
Evan Hunt -- e...@isc.org
Internet Systems Consortium, Inc.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

rndc valid key types

[Gregory Sloop]

So, I've spent some time looking at the man pages and googling without any 
definitive answer.

I'm generating some new rndc keys for my bind9 config. (9.11.3 in this 
particular case, if it matters.)

rndc-confgen has quite a number of options for the key-type - but I'm not sure 
what BIND9 will handle for RNDC.

I've seen reports that only HMAC-MD5 is the only valid key type.

...

Just before posting this, I checked the RNDC man page and found this: 
[At least I saved myself some public embarrassment! :) ]
---
rndc communicates with the name server over a TCP connection, sending commands 
authenticated with digital signatures. In the current versions of rndc and 
named, the only supported authentication algorithms are HMAC-MD5 (for 
compatibility), HMAC-SHA1, HMAC-SHA224, HMAC-SHA256 (default), HMAC-SHA384 and 
HMAC-SHA512. They use a shared secret on each end of the connection. This 
provides TSIG-style authentication for the command request and the name 
server's response. All commands sent over the channel must be signed by a 
key_id known to the server.
---

Still, the root cause for my query
Is there any (security) reason/implications to use something "better" than MD5?

I'd lean toward something like HMAC-SHA256/384/512.

Perhaps there's a discussion somewhere I haven't found - and I'd be glad to be 
pointed to that, instead of taking someone's time re-typing a bunch of details. 
But I can't seem to find anything. 
I assume it might be easier to forge an update for rndc with an MD5 key, right? 
Is there any reason not to select the strongest - HMAC-SHA512?

Just wanting to be sure I understand the implications of any particular choice.

TIA
-Greg___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Brett Delmage]

On Tue, 7 Jul 2020, Tony Finch wrote:


Brett Delmage  wrote:

On Tue, 7 Jul 2020, Tony Finch wrote:


minimal-any yes;


Why only reduce and not eliminate?


The reason is a bit subtle. If an ANY query comes via a recursive
resolver, it is much better to give the resolver an answer so that it will
put an entry in its cache...


This is a very interesting and clear explanation. Thanks for taking the 
time to share this Tony. TIL :-)


Brett

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Tony Finch]

Brett Delmage  wrote:
> On Tue, 7 Jul 2020, Tony Finch wrote:
> >
> > minimal-any yes;
>
> Why only reduce and not eliminate?

The reason is a bit subtle. If an ANY query comes via a recursive
resolver, it is much better to give the resolver an answer so that it will
put an entry in its cache. The cache entry will stop more ANY queries from
being sent from the resolver to the upstream auth server, as long as its
TTL lasts.

If the auth server does not answer, or sends a REFUSED error, the resolver
is likely to retry, which increases worthless traffic rather than
suppressing it, and the resolver may decide the auth server is lame which
will cause knock-on problems for legitimate queries.

There are some scenarios where reflection attacks go through multiple
servers. If you can get cache entries into those servers then the
attack traffic gets suppressed closer to its source. There have been quite
a lot of attacks that work like this:

  * an ISP has a huge number of customers with crappy home routers, that
can act as open recursive resolvers

  * an arsehole decides to use these crappy home routers in a reflection /
amplification DDoS attack

  * the crappy home routers forward the attack queries to their ISP's
recursive servers; these recursive servers are legitimate and well
configured but suffer from bad client devices

  * the recursive servers resolve the queries against some third party
authoritative servers

If the recursive servers cache the responses, then the auth servers should
not be much affected by the attack: most of the traffic is answered from the
ISP caches, and maybe the home router caches if they have them.

But if the auth servers don't answer, or send REFUSED errors, then the
recursive servers are going to keep retrying queries, and thereby relay a
very large proportion of the attack traffic to the auth servers. Sadness
will follow.

Note that RRL does not help in this scenario, because from the auth
server's point of view the ISP resolvers are legitimate clients, which RRL
can observe from their retry behaviour. RRL is designed for attacks where
the spoofed queries go direct to the auth server, which is not happening
in this case.

When this happened to us (when my servers were the third party auth
servers) the DDoS attack was hitting a very large number of ISPs, so our
auth servers were getting ANY queries via huge numbers of recursive
servers. Extra unfortunately, the ANY response was too big to fit in UDP,
so all the resolvers were trying to query over TCP. And our auth servers
did not have enough TCP capacity to handle the load. Much sadness. (It
didn't take us offline because our off-site auth servers were differently
configured and able to keep answering.)

So I implemented minimal-any to stop it from happening again.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Fisher, German Bight: Westerly veering northwesterly 4 to 6, decreasing 3
later in south German Bight. Moderate, occasionally rough at first. Mainly
fair. Mainly good.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[@lbutlr]

On 07 Jul 2020, at 12:06, Michael De Roover  wrote:
> On 7/7/20 4:06 PM, Tony Finch wrote:
> 
>>  max-udp-size 1420;
>>  https://dnsflagday.net/2020/

> Interesting, I wasn't aware of this campaign. I don't know if I'm 
> knowledgeable enough on UDP to be able to make educated decisions on this 
> myself but I look forward to its eventual release.

The URL has a good explanation of this setting and it looks like 1420 is a more 
than adequate packet size. 

>From  the page:
An EDNS buffer size of 1232 bytes will avoid fragmentation on nearly all 
current networks. This is based on an MTU of 1280, which is required by the 
IPv6 specification, minus 48 bytes for the IPv6 and UDP headers.

Sunce 1420 is still well under the MTU on most connections (usually 1500, 
sometimes 1492) and well above the required, I suspect this is fine as well. 
I've gone ahead and added to to my named.conf with a comment linking to Tony's 
message.




-- 
"Are you pondering what I'm pondering?"
"I think so, Mr. Brain, but if the sun'll come out tomorrow, what's
it doing right now?"

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Brett Delmage]

On Tue, 7 Jul 2020, Shumon Huque wrote:


Cloudflare themselves now implement the "minimal any" behavior described
in this spec:

    https://tools.ietf.org/html/rfc8482



cloudflare.com.         3789    IN      HINFO   "RFC8482" ""


Gee, that's a pretty minimal answer! Thanks.___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Shumon Huque]

On Tue, Jul 7, 2020 at 2:21 PM Brett Delmage  wrote:

> On Tue, 7 Jul 2020, Tony Finch wrote:
>
> > Reduce the size of responses to ANY queries, which are a favourite tool
> of
> > amplification attacks. There's basically no downside to this one, in my
> > opinion, but I'm biased because I implemented it.
> >
> >   minimal-any yes;
>
> Why only reduce and not eliminate?
>
> Can ANY responses be disabled completely with an option?
>
> This article at cloudflare
> https://blog.cloudflare.com/deprecating-dns-any-meta-query-type/
> states that they have deprecated it because it wasn't being used. They
> should know! This was posted over 5 years ago, in 2015.
>

Cloudflare themselves now implement the "minimal any" behavior described
in this spec:

https://tools.ietf.org/html/rfc8482

Responding to ANY with NOTIMP, REFUSED, or unknown RCODEs, or not
responding at all results in undesirable follow-on behaviour from DNS
resolvers
(mostly aggressive retries).

Shumon.

---
$ dig @ns1.cloudflare.com. cloudflare.com. ANY

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54526
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;cloudflare.com.IN  ANY

;; ANSWER SECTION:
cloudflare.com. 3789IN  HINFO   "RFC8482" ""
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Brett Delmage]

On Tue, 7 Jul 2020, Tony Finch wrote:


Reduce the size of responses to ANY queries, which are a favourite tool of
amplification attacks. There's basically no downside to this one, in my
opinion, but I'm biased because I implemented it.

minimal-any yes;


Why only reduce and not eliminate?

Can ANY responses be disabled completely with an option?

This article at cloudflare
https://blog.cloudflare.com/deprecating-dns-any-meta-query-type/
states that they have deprecated it because it wasn't being used. They 
should know! This was posted over 5 years ago, in 2015.


Brett
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Michael De Roover]

On 7/7/20 4:06 PM, Tony Finch wrote:


An auth-only server can also be used for amplification attacks that use
its authoritative zones - these attacks don't have to use recursion.
There are a few ways to mitigate auth-only amplification attacks.

Response rate limiting is very effective. Start off by putting the
following in your options{} section, and look in the BIND ARM for other
directives you can put in the rate-limit{} section.

rate-limit {
responses-per-second 10;
};
That's a really useful option to have, I didn't know about this yet. It 
seems like that could take care of the brunt of amplification attacks 
already. Definitely going to add this in, thanks!

Set a maximum UDP packet size, to suppress fragmented packets. The DNS
flag day 2020 campaign will make this a standard setting. For a long time
I have used:

max-udp-size 1420;

https://dnsflagday.net/2020/

A downside of small UDP responses is more truncated packets and more
queries over TCP, but there are still more ways to reduce response size
which also reduce truncation.
Interesting, I wasn't aware of this campaign. I don't know if I'm 
knowledgeable enough on UDP to be able to make educated decisions on 
this myself but I look forward to its eventual release.

Reduce the size of responses to ANY queries, which are a favourite tool of
amplification attacks. There's basically no downside to this one, in my
opinion, but I'm biased because I implemented it.

minimal-any yes;


I've heard of these ANY queries being preferred for amplification 
attacks as well, since the responses are often so large... I don't think 
that there would be any downsides to this either, in fact I've never 
actually seen a legitimate application use it... Probably best to lock 
down indeed.



You can also reduce the size of other answers. In theory this option might
force resolvers to make more queries to get records that by default would
appear in the additional section, but I think in practice resolvers make
these queries anyway because of RFC 2181 trustworthiness logic, and
because applications (such as SMTP servers) find it easier to query
directly than use additional records. So on my auth servers I set:

minimal-responses yes;


Hmm, for the authoritative name servers this might be a good idea yeah.. 
Those are authoritative only (i.e. `recursion no`). So for clients 
querying those, the NS records served in the additional section at least 
should already be known to the client anyway... I mean that's why 
they're there to begin with, so they must already know that information 
from the DNS servers higher up the chain. And another query if needed, 
saves traffic either way I suppose.


Thanks a lot for the detailed reply, I really appreciate it :)

--
Met vriendelijke groet / Best regards,
Michael De Roover
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Tony Finch]

@lbutlr  wrote:
>
> > rate-limit { responses-per-second 10; };
>
> Does that apply to local queries as well (for example, a mail server may
> easily make a whole lot of queries to 127.0.0.1, and rate limiting it
> would at the very least affect logging and could delay mail if the MTA
> cannot verify DNS.

I don't recommend using response rate limiting on recursive servers.

The principle behind RRL is that auth servers are queried by resolvers
with caches, and a correctly-functioning cache will not repeat the same
query very frequently, so it is reasonable to apply a rate limit on the
auth servers.

Recursive servers, on the other hand, are often queried by stub resolvers
without caches. The query rate is then entirely driven by the application
workload, and you can't apply a rate limit on the recursive server without
causing serious trouble for the application.

It can be especially bad because traditional cacheless stub resolvers are
not good at error recovery, and when RRL hits, their retry strategy is
likely to increase the query rate observed by the server, making things
worse.

If you are running an oldskool multi-purpose server that is recursive for
its own daemons but authoritative for others, then you can use the
`rate-limit { exempt-clients }` option so that RRL doesn't apply to
recursive clients. But I wouldn't recommend a setup like this. (My auth
servers have their /etc/resolv.conf pointing at my recursive service.)

> Do these setting also need to be applied to the secondary servers?

The settings I described are for public authoritative servers, i.e ones
that appear in NS records. These servers can be primary or secondary (but
are usually secondary).

Secondary servers don't necessarily appear in NS records, and if they
don't they are less likely to be exposed to this kind of attack.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Southeast Iceland: Westerly or southwesterly, 3 to 5, becoming variable 3 or
less later in north. Moderate. Showers. Good.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Canadian registrars that properly support ipv6 and DNSSEC ?

[Brett Delmage]

Not quite on-topic, but consider this an essential element of making my 
BIND signing, authoritative server and name service work well.


Does anyone know of or ideally have experience with Canadian 
(CIRA-authorized) and ideally _Canadian-based_ .ca registrars that handle 
DNSSEC and ipv6 properly?


I've been using namespro.ca for years. They are friendly and responsive, 
but I still have to file a ticket to submit an ipv6 nameserver address or 
DS record. They have not moved ahead on this. Thankfully, these are not 
frequent activities.


It would be nice to be able to automate it with an API, especially with 
newer support for key rotation. I guess I could automate sending of an 
email request to support using a BIND hook... :-p


Brett
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Fun with nsudpate and ac1.nstld.com

[Tony Finch]

@lbutlr  wrote:
>
> The latest surprise was that dnssec-enable yes; is obsolete in Bind 9.16.

`dnssec-enable yes` has been the default since 2007, so that directive has
been useless for quite a long time :-) What changed in 9.16 is that you
now can't turn DNSSEC off. (Specifically, support for correctly serving
signed zones on authoritative servers, and support for DNSSEC-aware
clients of resolvers, whether or not any validation is happening.
`dnssec-validation` is a separate setting.)

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
individual and social justice
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[@lbutlr]

On 07 Jul 2020, at 08:06, Tony Finch  wrote:

Excellent post, and a nice summary of some best practices.

I have a couple of questions.

> Response rate limiting is very effective. Start off by putting the
> following in your options{} section, and look in the BIND ARM for other
> directives you can put in the rate-limit{} section.
> 
>   rate-limit { responses-per-second 10; };

Does that apply to local queries as well (for example, a mail server may easily 
make a whole lot of queries to 127.0.0.1, and rate limiting it would at the 
very least affect logging and could delay mail if the MTA cannot verify DNS.

Do these setting also need to be applied to the secondary servers?



-- 
What's another word for Thesaurus?

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Fun with nsudpate and ac1.nstld.com

[@lbutlr]

On 06 Jul 2020, at 17:59, Mark Andrews  wrote:
> Nsupdate can normally determine the name of the zone that has to be updated 
> so most of the time you don’t need to specify the zone.  There are a few 
> cases, like when adding delegating NS records or glue to the parent zone you 
> have to override the normal zone discovery procedure.

So if I were to try adding web2.example.com via nsupdate I could simply 

> update add web2.example.com 96400 IN CNAME www.covisp.net
> send

That's good to know, but I fear I will remember that and use it in cases where 
I do need to specify it and muck things up.

I change DNS settings so infrequently that each time it is almost like starting 
over, especially since the underlying software has changed as well. Also, I 
need better notes, which I am taking this time. (Most of the serials on the DNS 
files are more than two years old)

The latest surprise was that dnssec-enable yes; is obsolete in Bind 9.16. I've 
noticed no fallout from simply uncommenting it, so I assume it is either 
required now or implied with dnssec-validation set or auto-dnssec in the zone 
config.

I do have motivation to get all this nsupdate stuff square, however, as I want 
to move Letsencrypt to wildcard certs and that requires updating the DNS during 
the LE exchange.



-- 
Vi Veri Veniversum Vivus Vici

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Tony Finch]

Michael De Roover  wrote:
>
> Said friend said to me that he tested my authoritative name servers and
> found them to be not vulnerable. [snip] They do not respond to recursive
> queries. It appears that the test of whether a server is "vulnerable" or
> not has to do with this. The command used to test this was apparently
> "dig +short test.openresolver.com TXT @your.name.server".

OK, that iss all right and correct, but there is (of course) a bit more to
this issue.

As you already know, the most basic thing to avoid is not being an open
recursive server. Out of the box, BIND has a recursion ACL that only
allows queries from directly connected networks, so you won't have this
problem without making an explicit configuration mistake. Normally for an
authoritative-only server, you should set `recursion no` to lock it down
more tightly.

An auth-only server can also be used for amplification attacks that use
its authoritative zones - these attacks don't have to use recursion.
There are a few ways to mitigate auth-only amplification attacks.

Response rate limiting is very effective. Start off by putting the
following in your options{} section, and look in the BIND ARM for other
directives you can put in the rate-limit{} section.

rate-limit {
responses-per-second 10;
};

Especially if you have DNSSEC signed zones then there are a few extra
things you can do to reduce the size of your response packets, which
reduces the attacker's amplification factor, and makes you less likely to
be abused.

Set a maximum UDP packet size, to suppress fragmented packets. The DNS
flag day 2020 campaign will make this a standard setting. For a long time
I have used:

max-udp-size 1420;

https://dnsflagday.net/2020/

A downside of small UDP responses is more truncated packets and more
queries over TCP, but there are still more ways to reduce response size
which also reduce truncation.

Reduce the size of responses to ANY queries, which are a favourite tool of
amplification attacks. There's basically no downside to this one, in my
opinion, but I'm biased because I implemented it.

minimal-any yes;

You can also reduce the size of other answers. In theory this option might
force resolvers to make more queries to get records that by default would
appear in the additional section, but I think in practice resolvers make
these queries anyway because of RFC 2181 trustworthiness logic, and
because applications (such as SMTP servers) find it easier to query
directly than use additional records. So on my auth servers I set:

minimal-responses yes;

If you are signing zones with DNSSEC, consider doing an algorithm
rollover to ECDSA p256 (algorithm 13) because this has much smaller
signatures than RSA. Algorithm rollovers are not particularly easy,
because you need a good grasp of the DNSSEC key timing parameters and how
and when to swap over your DS records. (There used to be even more
gotchas, so it is getting easier, honest!)

Finally, there's the built-in _bind CHAOS view. This has very strict
response rate limiting by default, but if you want to be super careful
you can set `version none` and `hostname none` to lock it down further.
(I don't bother with this.)

Here endeth the brain dump.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Mull of Galloway to Mull of Kintyre including the Firth of Clyde and North
Channel: Variable, 2 to 4. Moderate at first near the Mull of Kintyre,
otherwise smooth or slight. Showers. Mainly good.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNS security, amplification attacks and recursion

[Stephane Bortzmeyer]

On Tue, Jul 07, 2020 at 03:00:13PM +0200,
 Michael De Roover  wrote 
 a message of 46 lines which said:

> The command used to test this was apparently "dig +short
> test.openresolver.com TXT @your.name.server".

ANY instead of TXT may be more efficient (specially with +dnssec), if
the goal is to get the maximum amplification. Of course, if the server
implements RFC 8482, ANY won't help.

> Authoritative name servers may not need a huge DNS infrastructure
> for a small-ish zone (say under 1k records), but recursors on the
> scale of Google and Cloudflare in particular (not sure how popular
> Quad9 is so far).. those use massive infrastructure including
> anycast and everything! I'd consider it safe to assume that their
> servers are at least on the order of 100Gbps cumulatively, if not
> more.

This is precisely what makes them dangerous. They are good reflectors
(good from the point of view of the attacker). On the other hand, they
typically implement various forms of rate-limiting, and they are
monitored closely by knowledgeable professionals so, they may not be
good reflectors after all.

> If these would be vulnerable to amplification attacks just because
> they allow recursion,

They're not vulnerable, this attack works by reflection (just like the
NTP attack you mentioned) so they are not the potential victims, they
could be used as helpers.



___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

DNS security, amplification attacks and recursion

[Michael De Roover]

Hello,

Recently I discussed with a friend of mine the idea of NTP and DNS in 
the context of denial of service attacks. In NTP this amplification 
attack is done with the monlist command (that should honestly never have 
been publicly available due to its purpose being pretty much entirely 
debugging-related). The DNS version was rather unclear to me however.


Said friend said to me that he tested my authoritative name servers and 
found them to be not vulnerable. I don't run the latest and greatest of 
BIND at all, I mean it's Debian distribution packages we're talking 
about there... But they were set up to be exclusively authoritative. 
They do not respond to recursive queries. It appears that the test of 
whether a server is "vulnerable" or not has to do with this. The command 
used to test this was apparently "dig +short test.openresolver.com TXT 
@your.name.server". That's simply a recursive query of what appears to 
be an arbitrary record to me.


This also meant that supposedly the recursive DNS servers from Google, 
Cloudflare and Quad9 were all considered vulnerable. I find this very 
hard to believe. Authoritative name servers may not need a huge DNS 
infrastructure for a small-ish zone (say under 1k records), but 
recursors on the scale of Google and Cloudflare in particular (not sure 
how popular Quad9 is so far).. those use massive infrastructure 
including anycast and everything! I'd consider it safe to assume that 
their servers are at least on the order of 100Gbps cumulatively, if not 
more. If these would be vulnerable to amplification attacks just because 
they allow recursion, wouldn't skids be jumping on this like there's no 
tomorrow? It doesn't make any sense to me.


This seems to be not very well documented online (or more likely my 
search terms aren't right), so yeah... I wonder why the idea of 
recursion became associated with a vulnerable server in the first place.


--
Met vriendelijke groet / Best regards,
Michael De Roover
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

ISC funds the development of this software with paid support subscriptions. 
Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users