Re: [dnsdist] dnsdist 1.9.4 segfault

[Remi Gacogne via dnsdist]

Hi,

On 03/06/2024 09:39, Remi Gacogne wrote:

On 02/06/2024 22:43, Nicolas Baumgarten via dnsdist wrote:
we started testing 1.9.4 on centos 7 before upgrading, and it 
segfaults, after 20 seconds of running.


Thanks a lot for reporting this issue! I'll try to reproduce this on 
CentOS 7 later today, as it doesn't happen in my development setup or in 
our CI. In the meantime, would you by any chance be able to get a 
backtrace ("bt full" in gdb) from a core dump, if any has been 
generated? Or, if you still have the setup laying around and can spare a 
few minutes, by starting dnsdist inside gdb: "gdb dnsdist" then "run", 
and once you get to the crash "bt full".


In case anyone else stumbles upon this, the issue seems to be caused by 
a bug in the net-snmp version shipped by EL7. I can reproduce it by 
opening a very large number of file descriptors before starting the SNMP 
thread, which causes the net-snmp provided "run_alarms" function to 
segfault because it uses a deprecated net-snmp function that does not 
support large file descriptors (hence the warning in the logs).
I tried to reproduce this behaviour with a more recent version of 
net-snmp (5.9.4) and I couldn't, so for now I don't think I'll bother 
spending more time on it.
If you are running into this issue, please check how many TCP worker 
threads you are starting with setMaxTCPClientThreads, as it really 
doesn't make sense to start more than a dozen of them in most cases 
since 1.4.0 and legacy configurations sometimes use a very high number 
there.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist 1.9.4 segfault

[Remi Gacogne via dnsdist]

Hi!

On 02/06/2024 22:43, Nicolas Baumgarten via dnsdist wrote:
we started testing 1.9.4 on centos 7 before upgrading, and it segfaults, 
after 20 seconds of running.


Thanks a lot for reporting this issue! I'll try to reproduce this on 
CentOS 7 later today, as it doesn't happen in my development setup or in 
our CI. In the meantime, would you by any chance be able to get a 
backtrace ("bt full" in gdb) from a core dump, if any has been 
generated? Or, if you still have the setup laying around and can spare a 
few minutes, by starting dnsdist inside gdb: "gdb dnsdist" then "run", 
and once you get to the crash "bt full".


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Matching DNS server IP in request

[Remi Gacogne via dnsdist]

Hi Aleš,

On 17/05/2024 18:17, Aleš Rygl via dnsdist wrote:
     I would need to virtualy split a single dnsdist instance in the the 
way, that clients sendig request to a particular IP od DNS dnsdist 
(listening on multiple IPs) are sent to a dedicated pool. I could start 
another dnsdist process bound to the secondary IP on the server but I 
think there must be a more lelegant wa to do it ;-) It should work this 
way:


     client1 ---> DNS req. to dnsdist IP1 ---> pool-A ---> cache-A ---> 
backend A
     client2 ---> DNS req. to dnsdist IP2 ---> pool-B ---> cache-B ---> 
backend B


     Is there a way to match a destination IP of the client's request in 
a rule in order to send it to a pool? I can not use client's IP here.


Unless I'm mistaken you should be able to do that with NetmaskGroupRule 
[1], setting the "src" parameter to false to match the destination 
instead of the source.


[1]:
https://dnsdist.org/reference/selectors.html?highlight=netmaskgroup#NetmaskGroupRule

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.4 released

[Remi Gacogne via dnsdist]

Hello!

We released PowerDNS DNSdist 1.9.4 today. This release fixes 
CVE-2024-25581, a denial of service security issue affecting versions 
1.9.0, 1.9.1, 1.9.2 and 1.9.3 only. Earlier versions are not affected.


When incoming DNS over HTTPS support is enabled using the nghttp2 
provider, and queries are routed to a tcp-only or DNS over TLS backend, 
an attacker can trigger an assertion failure in DNSdist by sending a 
request for a zone transfer (AXFR or IXFR) over DNS over HTTPS, causing 
the process to stop and thus leading to a Denial of Service.


DNS over HTTPS is not enabled by default, and backends are using plain 
DNS (Do53) by default.


Two work-arounds are available:
- refuse incoming XFR requests via a DNSdist rule: 
addAction(OrRule({QTypeRule(DNSQType.AXFR), QTypeRule(DNSQType.IXFR)}), 
RCodeAction(DNSRCode.REFUSED))
- switch to the legacy h2o provider by setting library='h2o' in the 
addDOHLocal directive


We would like to thank Daniel Stirnimann from Switch for finding and 
subsequently reporting this issue.


This release also includes a few other fixes:
- Fix DNS over plain HTTP broken by reloadAllCertificates()
- Fix a crash in incoming DoH with nghttp2 when the incoming query is 
forwarded to the backend over TCP and the response comes back 
immediately. This issue was independently reported by Daniel Stirnimann 
from Switch and Stéphane Bortzmeyer, many thanks to them.

- Fix "C++ One Definition Rule" warnings in XSK

Please see the DNSdist website [1] for the more complete changelog [2] 
and the current documentation. The upgrade guide is also available there 
[3].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.9.4
[3]: https://dnsdist.org/upgrade_guide.html
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.9.4.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.9.4.tar.bz2.sig
[7]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist tuning for high qps on nxdomain ddos

[Remi Gacogne via dnsdist]

Hi!

On 03/05/2024 22:20, Jasper Aikema via dnsdist wrote:
Currently we are stuck at a max of +/- 200k qps for nxdomain requests 
and want to be able to serve +/- 300k qps per server.


200k QPS is fairly low based on what you describe. Would you mind 
sharing the whole configuration (redacting passwords and keys, of 
course), and telling us a bit more about the hardware dnsdist is running on?



We have done the following:
- added multiple (6x the amount of cores) addLocal listeners for IPv4 
and IPv6, with the options reusePort=true and tcpFastOpenQueueSize=100

> - add multiple (2x the amount of cores) newServer to the backend, with
> the options tcpFastOpen=true and sockets=(2x the amount of cores)

6 times the amount of cores is probably not a good idea. I usually 
advise to make it so that the number of threads is roughly equivalent to 
the number of cores that are dedicated to dnsdist, so in your case the 
number of addLocal + the number of newServer + the number of TCP workers 
should ideally match the number of cores you have. If you need to 
overcommit the cores a bit that's fine, but keep it to something like 
twice the number of cores you have, not 10 times.



- setMaxTCPClientThreads(1000)
I'm pretty sure this does not make sense, I would first go with the 
default until you see TCP/DoT connections are not processed correctly.


And the defaults like caching requests (which doesn't work for nxdomain) 
and limit the amount of qps per ip (which also doens't work for nxdomain 
attack because they use public resolvers).


When you say it doesn't work for NXDomain, I'm assuming you mean it 
doesn't solve the problem of random sub-domains attacks, not that a 
NXDomain is not properly cached/accounted?
When we simulate a nxdomain attack (with 200k qps and 500MBit of 
traffic) , we get a high load on the dnsdist server (50% CPU for dsndist 
and a lot of interrupts and context switches).


I expect lowering the number of threads will reduce the context switches 
a lot. If you are still not getting good QPS numbers, I would suggest 
checking if disabling the rules help, to figure out the bottleneck. You 
might also want to take a look with "perf top -p " 
during the high load to see where the CPU time is spent.



So the question from me to you are:
- how much qps are you able to push through dnsdist using a powerdns or 
bind backend


It really depends on the hardware you have and the rules you are 
enabling, but it's quite common to see people pushing 400k+ QPS on a 
single DNSdist without a lot of fine tuning, and a fair amount of 
remaining head-room.


- have I overlooked some tuning parameters, e.g. more kernel parameters 
or some dnsdist parameters


I shared a few parameters a while ago: [1].

- what is the best method of sending packets for a domain to a seperate 
backend, right we now we use 'addAction("", 
PoolAction("abuse")), but is this the least CPU intensive one? Are there 
better methods?


It's the best method and should be really cheap.

> I have seen eBPF socket filtering, but as far as I have seen that is 
for dropping unwanted packets.


Correct. You could look into enabling AF_XDP / XSK [2] but I would 
recommend checking that you really cannot get the performance you want 
with normal processing first, as AF_XDP has some rough edges.


[1]: https://mailman.powerdns.com/pipermail/dnsdist/2023-January/001271.html
[2]: https://dnsdist.org/advanced/xsk.html

Best regards,
--
Remi Gacogne
PowerDNS B.V


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.3 released

[Remi Gacogne via dnsdist]

Hello!

Less than an hour after the release of PowerDNS DNSdist 1.9.2 today, we 
received reports of DNSdist crashing in some setups. This 1.9.3 release 
fixes the issue that was introduced in 1.9.2, for now by reverting the 
related change.


Please see the DNSdist website [1] for the changelog [2] and the current 
documentation. The upgrade guide is also available there [3].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.9.3
[3]: https://dnsdist.org/upgrade_guide.html
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.9.3.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.9.3.tar.bz2.sig
[7]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] 1.9.2 crashing in tcpClient

[Remi Gacogne via dnsdist]

Hi Holger,

Thanks for reaching out. We have had another report already and are 
looking into it. We have already confirmed that reverting a recent 
change fixes it [1], and we will release 1.9.3 in a couple hours.


[1]: https://github.com/PowerDNS/pdns/pull/14040/files

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.2 released

[Remi Gacogne via dnsdist]

Hello!

We released PowerDNS DNSdist 1.9.2 today. This release fixes several issues:
- HTTP/1.1 was wrongly selected over HTTP/2 when a DNS over HTTPS client 
advertised both HTTP versions in ALPN and listed HTTP/1.1 first, and the 
nghttp2 provider was used

- The first connection to the DNSdist console done over IPv6 was rejected
- A failure of the first lazy health-check was not properly handled
- A crash might have occurred if an incoming DNS over HTTPS connection 
timed out right before the corresponding outgoing query to a backend 
did, and the nghttp2 provider was used
- DNS over HTTPS connections and queries counters were not working 
properly with the nghttp2 provider
- Incoming TCP connections from a client were not always closed right 
away after an error
- Outgoing TCP connections to a backend were not always closed right 
away after a timeout
- The Docker image was printing the DNSdist configuration to the 
terminal by default, including secrets, which might not have been expected
- It was not possible to return a "no server available" result from a 
custom Lua FFI load-balancing policy

- Several compilation warnings have been fixed

Please see the DNSdist website [1] for the more complete changelog [2] 
and the current documentation. The upgrade guide is also available there 
[3].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.9.2
[3]: https://dnsdist.org/upgrade_guide.html
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.9.2.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.9.2.tar.bz2.sig
[7]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Re: PowerDNS DNSdist 1.9.1 released

[Remi Gacogne via dnsdist]

Hi,

On 19/03/2024 16:05, Markus Ueberall wrote:
would it be possible to also provide arm64 binaries (and, maybe within a 
year, riscv64 binaries as well)? IMHO, this would save a growing number 
of people the hassle of rebuilding/repackaging the binaries themselves 
locally (and also help in case of error reports as everyone would be 
able to use the same build artefacts).


We are actually in the process of evaluating several options to do just 
that, but I don't have an ETA at this point.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] DoH issues after 1.8.3 -> 1.9.0 upgrade

[Remi Gacogne via dnsdist]

Hi,

On 18/03/2024 22:00, Christoph via dnsdist wrote:
This might be related:https://github.com/PowerDNS/pdns/issues/13850, 
not backported yet


thanks for the pointer, really looking forward to the dnsdist version
that has this solved.


Sure, I expect to release 1.9.2 including this fix in the next couple weeks.


The new nghttp2 provider for
incoming DNS over HTTPS does not support HTTP/1.1. In 1.9.x it's
still possible to switch back to the legacy h2o provider but note
that it will likely go away in the next major version of DNSdist. In
our testing the lack of HTTP/1.1 support was not an issue for actual
DNS over HTTPS clients, with most of HTTP/1.1 queries coming from
crawlers/bots, but of course we will reconsider if you find out that
legitimate DoH clients are impacted.


we see about 5-10% of non-version 2 DoH requests by looking at:

sum by (version)
(irate(dnsdist_frontend_doh_http_version_queries{job="$job"}[$__rate_interval]))


Note that this metric (doh_http_version_queries) is incremented after 
doing some sanity checks but before actually parsing the DNS query, so 
unfortunately we cannot be sure these are valid DoH queries. At this 
point they could be bots. Can you check doh_version_status_responses for 
httpversion=1 and status=200 instead?



So the practical solution to use dnsdist 1.9.0 with nghttp2 and
still support HTTP/1.1 clients is to use a webserver like nginx in front 
of dnsdist?


Yes, a reverse proxy like nginx or HAProxy might be the best option to 
keep HTTP/1.1 support at this point.



I expected an increase of this metric during our partial outage but
this value did not increase, is this expected?

irate(dnsdist_frontend_doh_version_status_responses{httpversion="1",status="400",job="$job"}[$__rate_interval])

dnsdist_frontend_noncompliantqueries also didn't increase.
Which value is expected to increase?


I'm afraid we are currently not increasing any counter in this exact 
case, I'll see what I can do about it.



btw:
dnsdist's v1.9.0 answer to HTTP requests not using HTTP/2:


This server implements RFC 8484 - DNS Queries over HTTP, and
requires HTTP/2 in accordance with section 5.2 of the RFC.


but RFC8484 does not actually require HTTP/2, right?

https://www.rfc-editor.org/rfc/rfc8484.html#section-5.2
 > 5.2.  HTTP/2


HTTP/2 [RFC7540] is the minimum RECOMMENDED version of HTTP for use 
with DoH.


It is recommended but not a "MUST".


You are correct, but in practice I am yet to see a DoH client using 
HTTP/1.1 in production. Bind 9, Unbound and Knot also only support DNS 
over HTTP/2. That being said, I'm really open to implementing DNS over 
HTTP/1.1 if it serves a real purpose, I just don't want to increase the 
code complexity and attack surface just to reply to crawlers..


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] DoH issues after 1.8.3 -> 1.9.0 upgrade

[Remi Gacogne via dnsdist]

Hi Christoph,

In addition to the issue mentioned by Otto, it might also be that the 
monitoring does not support HTTP/2. The new nghttp2 provider for 
incoming DNS over HTTPS does not support HTTP/1.1. In 1.9.x it's still 
possible to switch back to the legacy h2o provider but note that it will 
likely go away in the next major version of DNSdist. In our testing the 
lack of HTTP/1.1 support was not an issue for actual DNS over HTTPS 
clients, with most of HTTP/1.1 queries coming from crawlers/bots, but of 
course we will reconsider if you find out that legitimate DoH clients 
are impacted.


Best regards,

Remi

On 17/03/2024 19:12, Otto Moerbeek via dnsdist wrote:

On Sun, Mar 17, 2024 at 06:41:13PM +0100, Christoph via dnsdist wrote:


Hi,

in February we upgraded our test DoH/DoT server from 1.8.3 to 1.9.0
but we did not notice any problems so we upgraded our production server
from 1.8.3 to 1.9.0 yesterday.

Immediately after upgrading our monitoring claimed our DoH service is
unavailable (HTTP 400) but we were unable to reproduce it using firefox.

A closer look confirmed that there is some issue because we see about 50%
less DoH requests in our grafana graphs showing DoH request rates.

Having a look at the request rates per HTTP method suggests that we "loose"
almost all GET requests but also a significant fraction of POST DoH
requests.

sum by (method) 
(irate(dnsdist_frontend_doh_http_method_queries{job="$job"}[$__rate_interval]))

After looking at the TLS versions graph I noticed a clear correlation
but then I realized that all our DoH requests are TLS version 1.3
because we set minTLSVersion='tls1.3' - so this might be irrelevant.

irate(dnsdist_frontend_tlsqueries{job="$job"}[$__rate_interval])

2024-03-16 20:57:59 dnsdist upgraded: 1.8.3_1 -> 1.9.0
2024-03-16 20:59 monitoring says DoH is down (HTTP 400 - Bad Request)
monitoring requests this: 
https://doh.applied-privacy.net/query?dns=l1sBAAABA3d3dw1rbm90LXJlc29sdmVyAmN6AAAcAAE
Mar 17 02:40:45 bender-dpriv1 kernel: pid 77544 (dnsdist), jid 0, uid 208:
exited on signal 11 -> also interesting put likely unrelated?

Today we downgraded to 1.8.3, and everything went back to normal.

Is anyone else observing similar issues on dnsdist 1.9.0?

DoT does not appear to be affected.

best regards,
Christoph

OS: FreeBSD 13.2
dnsdist installed via pkg

our dnsdist config:

newServer({address="109.70.100.136", maxInFlight=1000, sockets=32,
name="clamps"})
newServer({address="109.70.100.140", maxInFlight=1000, sockets=32,
name="roberto"})
--newServer({address="109.70.100.133", sockets=4, name="titanius-dpriv1"})
setServerPolicy(leastOutstanding)

addTLSLocal("0.0.0.0",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.crt",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.key", 
{ciphers='ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256',
minTLSVersion='tls1.2', tcpFastOpenQueueSize=1000, maxInFlight=1000 })
addTLSLocal("[::]",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.crt",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.key", 
{ciphers='ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256',
minTLSVersion='tls1.2', tcpFastOpenQueueSize=1000, maxInFlight=1000 })

addDOHLocal("0.0.0.0:444",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.crt",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.key",
"/query", {minTLSVersion='tls1.3', serverTokens='doh',
tcpFastOpenQueueSize=1000, tcpListenQueueSize=4096 })
addDOHLocal("[::]:444",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.crt",
"/usr/local/etc/ssl/lego/certificates/doh.applied-privacy.net.key",
"/query", {minTLSVersion='tls1.3', serverTokens='doh',
tcpFastOpenQueueSize=1000, tcpListenQueueSize=4096 })

setACL({'0.0.0.0/0', '::/0'})
controlSocket('127.0.0.1:5199')
setConsoleACL('127.0.0.1/8')

setKey()

pc = newPacketCache(5, {maxTTL=86400, minTTL=3, temporaryFailureTTL=60,
staleTTL=60, dontAge=false})
getPool(""):setCache(pc)

webserver("127.0.0.1:8083")
setWebserverConfig({...})
setVerboseHealthChecks(true)
addAction(QTypeRule(65535), RCodeAction(DNSRCode.NOTIMP))



This might be related: https://github.com/PowerDNS/pdns/issues/13850,
not backported yet

-Otto

___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist




OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.1 released

[Remi Gacogne via dnsdist]

Hello!

We released PowerDNS DNSdist 1.9.1 today. This version brings no 
functional changes, and only bumps the version of the Quiche library we 
use, to incorporate a recent security update [1], fixing CVE-2024-1410 
[2] and CVE-2024-1765 [3]. This applies only if you configured incoming 
DoQ or DoH3 using the addDOQLocal() or addDOH3Local() functions. If you 
are not using our packages but are building DNSdist from the source code 
with DoQ or DoH3 support enabled, you will have to make sure your 
version of Quiche has been upgraded to 0.20.1.


Please see the DNSdist website [4] for the more complete changelog [5] 
and the current documentation. The upgrade guide is also available there 
[6].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [7].


The release tarball [8] and its signature [9] are available on the 
downloads website, and packages for several distributions are available 
from our repository [10].


[1]: https://github.com/cloudflare/quiche/releases/tag/0.20.1
[2]: https://www.cve.org/CVERecord?id=CVE-2024-1410
[3]: https://www.cve.org/CVERecord?id=CVE-2024-1765
[4]: https://dnsdist.org
[5]: https://dnsdist.org/changelog.html#change-1.9.1
[6]: https://dnsdist.org/upgrade_guide.html
[7]: https://github.com/PowerDNS/pdns/issues/new/choose
[8]:
https://downloads.powerdns.com/releases/dnsdist-1.9.1.tar.bz2
[9]:
https://downloads.powerdns.com/releases/dnsdist-1.9.1.tar.bz2.sig
[10]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.0

[Remi Gacogne via dnsdist]

Hello!

We are very happy to release PowerDNS DNSdist 1.9.0 today! This new 
version brings a fair number of new features since 1.8.3:


- DNS over QUIC [1]
- DNS over HTTP3
- AF_XDP [2] support
- the ability to set Extended DNS Error [3] statuses
- a cache-miss ratio dynamic block rule
- getAddressInfo for asynchronous DNS resolution
- Proxy Protocol support for TeeAction
- Proxy Protocol support can now be enabled on a per-bind basis
- many new selectors and actions

We would like to express our gratitude to Y7n05h [4] who contributed 
AF_XDP support during Google Summef Code! It took us far too long to 
integrate their contribution into a release, but it's finally there with 
impressive results.


We also replaced the default library handling DNS over HTTPS, switching 
from h2o to nghttp2 [5]. This change should be transparent for most 
users, since we made sure to preserve the existing features and 
configuration directives. Switching to nghttp2 allows us to support 
hardware acceleration for TLS exchanges, using for example Linux's kTLS 
[6] or Intel Quick-Assist Technology [7]. It also reduces our footprint 
on low-end devices by not requiring an additional library, since nghttp2 
was already used for outgoing DNS over HTTPS requests. Finally, while it 
was a long time coming, h2o is officially [8] no longer maintained in a 
way that makes it possible to use it as a stable library. Technically it 
will still be possible to revert to the use of h2o for incoming DNS over 
HTTPS in DNSdist 1.9.x, but we will remove that support after that.


Packagers need to be aware that SNMP support is no longer enabled by 
default, as it had been causing integration issues in some environments 
for a while, but it's still enabled in our packages. Two new features, 
DNS over QUIC and DNS over HTTP3, require the Cloudflare's Quiche [9] 
library, which is written in Rust [10] and might not be already present 
in some distributions.


We also made changes to our Open Source End of Life policy. Older 
release trains are now supported for one year after the following major 
release. Consult the EOL policy [11] for more details.


Please see the DNSdist website [12] for the more complete changelog [13] 
and the current documentation. The upgrade guide is also available there 
[14].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [15].


We are grateful to the PowerDNS community for the reporting of bugs, 
issues, feature requests, and especially to the submitters of fixes and 
implementations of features. We are particularly thankful to Denis 
Machard for testing and reporting issues with dnstap and protobuf 
exports, Håkan Lindqvist for tirelessly tracking issues in our DNS over 
HTTP3 feature, Oto Šťáva from the Knot Resolver team for testing DNSdist 
against his DNS over QUIC implementation in DNS Shotgun and reporting 
several discrepancies!


The release tarball [16] and its signature [17] are available on the 
downloads website, and packages for several distributions are available 
from our repository [18].


[1]: https://www.rfc-editor.org/rfc/rfc9250.html
[2]: https://www.kernel.org/doc/html/next/networking/af_xdp.html
[3]: https://www.rfc-editor.org/rfc/rfc8914.html
[4]: https://github.com/Y7n05h
[5]: https://nghttp2.org/
[6]: https://docs.kernel.org/networking/tls-offload.html
[7]: 
https://www.intel.com/content/www/us/en/architecture-and-technology/intel-quick-assist-technology-overview.html

[8]: https://github.com/h2o/h2o/issues/3230
[9]: https://github.com/cloudflare/quiche
[10]: https://www.rust-lang.org/
[11]: https://dnsdist.org/eol.html
[12]: https://dnsdist.org
[13]: https://dnsdist.org/changelog.html#change-1.9.0
[14]: https://dnsdist.org/upgrade_guide.html
[15]: https://github.com/PowerDNS/pdns/issues/new/choose
[16]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0.tar.bz2
[17]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0.tar.bz2.sig
[18]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Empty NOERROR being sent when backend times out

[Remi Gacogne via dnsdist]

Hi,

On 09/02/2024 11:05, Adam Bishop via dnsdist wrote:
I'm seeing an issue where caching resolvers outside of our network are 
occasionally storing empty responses to queries.


I think what's happening is that when a query is made and there's a 
backend timeout, dnsdist is responding to the user with an empty answer 
and NOERROR. Messages about a backend beign marked as down are in the 
log coinciding with when this has happened.


dnsdist cannot generate a response from a timeout, it simply does not 
respond at all.
It can however generate a SERVFAIL if there is no backend available when 
setServFailWhenNoServer [1] is set, which is not the default, but the 
backends need to be marked as unavailable when the query comes in, 
dnsdist will not generate a response once the query has been forwarded 
to a backend.


I've not caught dnsdist in the act yet with a packet capture as the 
issue is infrequent, but am I on the right track?


Is it possible to make dnsdist respond with a SERVFAIL for a backend 
timeout?


Nope.

[1]: https://dnsdist.org/guides/serverselection.html#setServFailWhenNoServer

Hope that helps,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.0-rc1 released

[Remi Gacogne via dnsdist]

Hello!

We are excited to release the first release candidate of what will 
become PowerDNS DNSdist 1.9.0!


The latest addition to DNSdist is AF_XDP[1] support. AF_XDP is a Linux 
feature optimized for high performance packet processing, allowing 
DNSdist to process UDP datagrams even faster than it already was. To 
give you a concrete example, on an Intel E3-1270 with 4 cores (8 
threads) running at 3.8 Ghz, with a 10 Gbps Intel 82599 network card, 
DNSdist can reply with a static answer at 1 million UDP queries per 
second without AF_XDP, and around 2.5 millions with AF_XDP. Of course 
this comes with some limitations, requires a recent Linux kernel and 
support in the network device driver. Please refer to our documentation 
[2] to know more.


In addition to this new feature, our DNS over QUIC and DNS over HTTP3 
implementations have been significantly improved and several bugs fixed. 
We are particularly thankful to Denis Machard for testing and reporting 
issues with dnstap and protobuf exports, Håkan Lindqvist for tirelessly 
tracking issues in our DNS over HTTP3 feature, Oto Šťáva from the Knot 
Resolver team for testing DNSdist against his DNS over QUIC 
implementation in DNS Shotgun and reporting several discrepancies!


Please see the DNSdist website [3] for the more complete changelog [4] 
and the current documentation. The upgrade guide is also available there 
[5].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [6].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [7] and its signature [8] are available on the 
downloads website, and packages for several distributions are available 
from our repository [9].


[1]: https://www.kernel.org/doc/html/next/networking/af_xdp.html
[2]: https://dnsdist.org/advanced/xsk.html
[3]: https://dnsdist.org
[4]: https://dnsdist.org/changelog.html#change-1.9.0-rc1
[5]: https://dnsdist.org/upgrade_guide.html#x-to-1-9-0-rc1
[6]: https://github.com/PowerDNS/pdns/issues/new/choose
[7]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-rc1.tar.bz2
[8]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-rc1.tar.bz2.sig
[9]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] AW: Suggestions for rules to block abusive traffic

[Remi Gacogne via dnsdist]

On 09/01/2024 09:50, Klaus Darilion wrote:

I fully agree, and we are working on having smarter mitigations in
dnsdist to only drops/truncate/route to a different pool queries that
are very likely to be part of a PRSD/enumeration attack.


Do you already have ideas how to implement that? I have thought a lot about an algorithm 
to block only "bad" queries bad have not found a method yet.


We have been looking into several heuristics, like the entropy of the 
queries, and we are getting good results.



For authoritative nameservers, meanwhile I think it would be better to just 
load the attacked zone completely into dnsdist or pdns-cache (or something 
similar to aggressive caching). Because I think just answering (mostly 
NXDOMAIN) may be faster then deciding if a query is bad or good.


We have already deployed something like that for zones that are not 
DNSSEC-signed: dnsdist learns the content of the zone via XFR, send 
NXDOMAIN for names that do not exist and pass the remaining ones to the 
backend. I know some people have done it in a different way and load 
attacked zones into a LMDB PowerDNS, telling dnsdist to route queries 
for these zones to the LMDB PowerDNS server. Of course most of the 
difficulty lies in automated this, which is very specific to every setup.


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] AW: Suggestions for rules to block abusive traffic

[Remi Gacogne via dnsdist]

Hi!

On 08/01/2024 23:08, Klaus Darilion wrote:

This is unfortunately a common issue indeed these days. It is possible
to use dnsdist to detect and mitigate these attacks to a certain extent,
using the StatNode API along with DynBlockRulesGroup:setSuffixMatchRule
[1] or the FFI equivalent for better performance. It requires writing a
bit of Lua code and some tuning on top of dnsdist, but all the building
blocks are there already. We have implemented this for several customers
and they are happy with the results.


How does this work in detail? Does your implementation block only the queries for 
.example.com or also "normal" queries like www.example.com or 
example.com MX? Or do you explicitly allow common subdomains before blocking everything else?


It really depends on the actual implementation in Lua. Currently when 
DynBlockRulesGroup:setSuffixMatchRule() is used it will insert a dynamic 
block for the suffix that is detected as being attacked, which will 
indeed apply to "normal" queries like www.example.com or example.com MX 
as well, although it's possible to allow-list specific suffixes, or to 
prevent blocking suffixes with not enough labels, for example.
We will be implementing the ability to instead route the detected suffix 
to a different pool soon, as suggested by Jacob in [1].



Blocking all queries to the attacked domain prevents collateral damage, but 
causes a DoS to the attacked domain and makes the customer of the attacked 
domain unhappy.


I fully agree, and we are working on having smarter mitigations in 
dnsdist to only drops/truncate/route to a different pool queries that 
are very likely to be part of a PRSD/enumeration attack.
Of course it's easier when the backend can handle the load, which is one 
of the reasons why the LMDB backend has been implemented, along with 
lightningstream :)


[1]: https://github.com/PowerDNS/pdns/issues/13374

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Suggestions for rules to block abusive traffic

[Remi Gacogne via dnsdist]

Hi Dan,

On 08/01/2024 17:28, Dan McCombs via dnsdist wrote:

  In our case we are affected as we use Pdns + DB backend as backend.

Yep, that's exactly our case as well - our legacy Pdns + mysql backends 
don't handle this very well. Longer term we intend to move away from 
that, but finding some improvements in the meantime for handling these 
floods would be helpful. I'll let you know if we come up with anything 
interesting!


This is unfortunately a common issue indeed these days. It is possible 
to use dnsdist to detect and mitigate these attacks to a certain extent, 
using the StatNode API along with DynBlockRulesGroup:setSuffixMatchRule 
[1] or the FFI equivalent for better performance. It requires writing a 
bit of Lua code and some tuning on top of dnsdist, but all the building 
blocks are there already. We have implemented this for several customers 
and they are happy with the results.


Best regards,

[1]: 
https://dnsdist.org/reference/config.html#DynBlockRulesGroup:setSuffixMatchRule

--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.8.3 released

[Remi Gacogne via dnsdist]

Hello!

We are very happy to release PowerDNS DNSdist 1.8.3 today, a maintenance 
release fixing a few bugs reported since 1.8.2:


- The exponential back-off timer used when a carbon server is 
unreachable had a bug that could lead to a busy-loop, consuming CPU time 
until the carbon server became reachable again
- In some cases, truncated UDP responses from a backend were not 
properly discarded

- Removing the last rule of a rules chain by its name or UUID was broken
- Several cosmetic issues related to eBPF dynamic blocks were fixed
- The JSON produced by the REST API might have been invalid when some 
special characters were present


In addition to these fixes, two new Lua bindings were added: 
DynBlockRulesGroup:removeRange and DNSHeader:getTC.


Please see the DNSdist website [1] for the more complete changelog [2] 
and the current documentation. The upgrade guide is also available there 
[3].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.8.3
[3]: https://dnsdist.org/upgrade_guide.html#x-to-1-8-3
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.8.3.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.8.3.tar.bz2.sig
[7]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.0-alpha4 released

[Remi Gacogne via dnsdist]

Hello!

We are thrilled to release the fourth alpha release of what will become 
PowerDNS DNSdist 1.9.0!


The most exciting new feature in this latest alpha is support for DNS 
over HTTP/3! Like DNS over QUIC for which we announced support in the 
previous alpha, DNS over HTTP/3 uses QUIC to provide excellent 
performance in challenging environments. We are again leveraging 
Cloudflare's Quiche [1] for this new feature, keeping the number of 
DNSdist dependencies small.


We also added a few smaller features since alpha 3:

- support for setting Extended DNS Error statuses
- a cache-miss ratio dynamic block rule
- getAddressInfo for asynchronous DNS resolution
- a rings endpoint to the REST API
- NetmaskGroup:addNMG to merge Netmask groups
- an option to set the SSL proxy protocol TLV
- Proxy Protocol v2 support to TeeAction
- enabling incoming PROXY protocol on a per-bind basis
- the maximum size of entries in the packet cache is now configurable
- raw response spoofing for ANY queries
- QNameSuffixRule, PayloadSizeRule and TCResponseAction
- DynBlockRulesGroup:removeRange
- setting the action from setSuffixMatchRule's visitor is now supported
- we now send a HTTP 400 response to legacy HTTP/1.1 clients with nghttp2

And fixed a few issues:

- Kees Monshouwer removed legacy terms from the codebase
- building without DoH but with nghttp2 was broken
- Quiche detection did not properly check the version
- DNS over QUIC latency metrics were missing
- removing the last rule by its name or UUID was broken
- building with DNS over QUIC but without DNS over HTTPS or DNS over TLS 
was broken


Please see the DNSdist website [2] for the more complete changelog [3] 
and the current documentation. The upgrade guide is also available there 
[4].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [5].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [6] and its signature [7] are available on the 
downloads website, and packages for several distributions are available 
from our repository [8].


[1]: https://github.com/cloudflare/quiche
[2]: https://dnsdist.org
[3]: https://dnsdist.org/changelog.html#change-1.9.0-alpha4
[4]: https://dnsdist.org/upgrade_guide.html#x-to-1-9-0-alpha4
[5]: https://github.com/PowerDNS/pdns/issues/new/choose
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-alpha4.tar.bz2
[7]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-alpha4.tar.bz2.sig
[8]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Re: Question about implementing dynBlockRulesGroup

[Remi Gacogne via dnsdist]

Hi,

On 04/12/2023 14:37, CamZie wrote:
I tried testing "MaxQPSIPRule" by setting it to "3" but the drop 
connection only occurs on every 4th request. We would like to be able to 
block all requests from the source IP after they reach a certain limit.


Right, it allows 3 queries per second, so the 4th one is blocked, dig 
times out after a delay of one second, so there is again a 3-queries 
credit for the next second, and so on and so forth.


So am I correctly understanding that you want to block for a given 
amount of time a client after it has exceeded, once, a given QPS rate? 
The Dynamic Blocks are designed to do just that, but as discussed before 
there is a delay of up to one second before they are enforced for the 
first time, for performance reasons. If you cannot live with that delay, 
I'm afraid you will have to implement a custom rule using Lua [1].


[1]: https://dnsdist.org/advanced/luaaction.html

Hope that helps,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Compiling with DNS-over-QUIC on OpenSUSE - quiche not found

[Remi Gacogne via dnsdist]

Hi!

On 21/11/2023 13:37, Oto Šťáva via dnsdist wrote:

Hi, sorry about the delay. This seems to work, big thanks!


Thanks for reporting back, much appreciated!

One thing I noticed is that if I add '--enable-dns-over-quic' but not 
'--enable-dns-over-tls', it successfully configures, but fails to 
compile because of missing symbols. Perhaps there should be an 
additional check that TLS is enabled a pre-requisite of QUIC, or a 
change so that QUIC automatically implies TLS as well?


I can reproduce that, and this is indeed a bug. I'll get it fixed, thanks!

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] addAction OpCode Iquery

[Remi Gacogne via dnsdist]

Hi,

On 16/11/2023 04:37, Nicolas Baumgarten via dnsdist wrote:

Queries with opcode 1 (DNSOpcode.IQuery) are being ignored (droped?) on 1.4
But 1.6.1 answers NOT implemented.


My guess is that these queries have a query records count (qdcount) of 0 
and you are seeing the effect of [1], implemented in 1.6.0-alpha1, 
because it was needed to conform to rfc8906's tests.


We don't know which is the reason for this queries, but in the not 
implemented scenario these queries are retried for a couple of minutes, 
hundreds or thousands per second by some devices.


That's awful, and of course the device should be fixed, but 
unfortunately not unheard of.



Trying  to stop this, we created a rule to drop them but it's not working:
  addAction(OpcodeRule(DNSOpcode.IQuery),DropAction())
the same with opcode Query works.

#   Name                             Matches Rule   
                               Action
0                                          0 opcode==1   
                              no op
1                                     191722 opcode==0   
                              no op


There is some preprocessing before the rules which answers not implemented?


Correct, this check occurs very early, if only because several rules 
assume that all queries have a qname which is not true when qdcount == 0.

There is any option to solve this? If not, we will try with iptables.


Not at the moment, no. We could make the qdcount==0 behaviour 
configurable, to allow dropping or sending a custom response code 
(Refused? No Error?) instead of Not Implemented. Opening a feature 
request would go a long way to make it happen :)


[1]: https://github.com/PowerDNS/pdns/pull/9991

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] rmResponseRule("name") not working

[Remi Gacogne via dnsdist]

Hi!
On 13/11/2023 22:59, Holger Hoffstätte via dnsdist wrote:


I'm running 1.8.2 and have a response rule:


showResponseRules()

#   Name  Matches Rule   Action
0   myRule    501 All    Lua response 
script


Now I try to remove it by name:


rmResponseRule("myRule")

Error: no rule matched

Removing by index (0) works fine, but I only know the name.
If I understand the docs and the source code correctly, this should work.
Suggestion? Haven't started debugging yet, just wanted to make sure I'm
not doing something wrong.


It's a bug indeed! The code doesn't properly test if the requested rule 
has been found and bails out if the rule is the last one in the list.
I just opened a pull request [1] that fixes it for me. Thanks for 
reporting this issue!


[1]: https://github.com/PowerDNS/pdns/pull/13488

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Compiling with DNS-over-QUIC on OpenSUSE - quiche not found

[Remi Gacogne via dnsdist]

Hi Oto!

On 30/10/2023 14:06, Oto Šťáva via dnsdist wrote:
I wanted to do some testing with the new DNS-over-QUIC implementation in 
dnsdist on my OpenSUSE machine. Quite understandably, OpenSUSE does not 
ship alpha versions of dnsdist, so I opted to compile the new version 
from source.


I looked through the code to find out dnsdist uses Cloudflare's Quiche, 
which is also missing from OpenSUSE's repositories, and they do not seem 
to provide any good way of installing it from source into the system. 
Dnsdist contains a 'builder-support/helpers/install_quiche.sh' script, 
which runs successfully (or at least never complains about anything), 
but when I run './configure.sh --enable-dns-over-quic', I get the 
following message:


configure: error: DNS over QUIC support requested but quiche was not found

Is there something I'm missing here? Is it just some strange behaviour 
on OpenSUSE's side?


I have not tried yet, but I guess that the directory in which our 
install script installs the Quiche pkg-config module file 
(/usr/lib/pkgconfig) is not in the default pkg-config path on openSUSE. 
This is also the case on RH-based distributions where we had to add [1] 
/usr/lib/pkgconfig to PKG_CONFIG_PATH for Quiche to be detected.


Can you try setting 
PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig at the end of 
your configure line?


[1]: 
https://github.com/PowerDNS/pdns/blob/master/builder-support/specs/dnsdist.spec#L108

--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Question about implementing dynBlockRulesGroup

[Remi Gacogne via dnsdist]

Hi,

On 30/10/2023 11:08, CamZie via dnsdist wrote:
We would like to use DNSdist to block traffics that exceeds a QPS limit 
and we have configured the following as test:


local dbr = dynBlockRulesGroup()

dbr:setQueryRate(5, 1, "Exceeded query rate", 60)
dbr:setQTypeRate(DNSQType.ANY, 2, 1, "Exceeded ANY rate", 60)

function maintenance()
   dbr:apply()
end

However, when we do 10 queries with the following command, all 10 
requests still goes through successfully:


for a in {0..10}; do dig -t a  @ +short; done

 From the console, we can see that the client has been detected and is 
listed in the blocklist but still the 10 queries has gone through even 
though we have limited it to 5.



showDynBlocks()
What                      Seconds   Blocks Warning    Action 
   Reason
/32              56        0 false      Drop 
Exceeded query rate


This is expected, as 'maintenance' is called every second so it might 
take up to a second for the client to get blocked.


Is there a way we can immediately drop the connection after reaching max 
5 queries per second as defined in the config? This is the same case 
with the ANY requests restriction.


MaxQPSIPRule [1]should do that. It is a bit more expensive than dynamic 
blocks when you have a lot of queries per second because it has to 
update a state for every query, but the "shards" parameter added in 
1.8.0 should help a lot under heavy load.


[1]: https://dnsdist.org/rules-actions.html#MaxQPSIPRule

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.0-alpha3 released

[Remi Gacogne via dnsdist]

Hello!

We are thrilled to release the third alpha release of what will become 
PowerDNS DNSdist 1.9.0!


Let's first address the elephant in the room: the second alpha was never 
released due to a last-minute issue discovered in RPM packaging after 
the tag was pushed, so we went to alpha3 right away.


The most exciting new feature in this third alpha is support for DNS 
over QUIC [1], which combines the confidentiality and integrity 
capabilities of DNS over TLS and DNS over HTTPS without the overhead of 
TCP connections.
Our implementation is based on Cloudflare's Quiche [2], which has 
already been battle-tested by being used on their edge network and in 
Android's DNS resolver. We first selected Quiche as the building block 
for QUIC because the API is both simple and powerful, but also because 
it is written in Rust. Rust is a memory-safe language and significantly 
reduces the risk of security issues.
One annoying drawback is that Quiche has not yet been packaged in most 
Linux distributions. This is not an issue if you are using our packages, 
because we ship the latest release of Quiche along DNSdist, but it might 
make building DNSdist with DNS over QUIC support a bit harder if you are 
doing it on your own, as you will need to first compile Quiche. We hope 
that distributions will adopt Quiche in the near future.


In addition to DNS over QUIC, we also added a few new features:
- the ability to parse Extended DNS Errors present in responses and 
export them via protobuf
- Denis Machard added Lua bindings to look at the selected backend from 
Lua rules and actions


We also fixed a few issues:
- phonedph1 fixed a typo on the metric name for TCP client timeouts
- contrary to what we announced, h2o support was not available anymore 
in our packages in the first alpha
- incoming DoH connections were not using the proper timeout value when 
handled by nghttp2

- cosmetic issues in eBPF dynamic block reporting
- invalid subnet masks coming from a string were not properly normalized
- DNS header might have been misaligned in some cases, causing issues on 
some architectures

- some log messages were not recorded at the proper level

Please also note that, as we did for stable releases, we switched to our 
own fork of libh2o [3] in order to mitigate CVE-2023-44487 [4], also 
known as HTTP/2 rapid reset [5].


We still have a few surprises left for 1.9.0 final, but more on that later!

Please see the DNSdist website [6] for the more complete changelog [7] 
and the current documentation. The upgrade guide is also available there 
[8].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [9].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [10] and its signature [11] are available on the 
downloads website, and packages for several distributions are available 
from our repository [12].


[1]: https://www.rfc-editor.org/rfc/rfc9250.html
[2]: https://github.com/cloudflare/quiche
[3]: https://github.com/PowerDNS/h2o/tree/v2.2.6%2Bpdns
[4]: https://www.cve.org/CVERecord?id=CVE-2023-44487
[5]: 
https://blog.cloudflare.com/technical-breakdown-http2-rapid-reset-ddos-attack/

[6]: https://dnsdist.org
[7]: https://dnsdist.org/changelog.html#change-1.9.0-alpha3
[8]: https://dnsdist.org/upgrade_guide.html#x-to-1-9-0-alpha3
[9]: https://github.com/PowerDNS/pdns/issues/new/choose
[10]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-alpha3.tar.bz2
[11]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-alpha3.tar.bz2.sig
[12]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Is it possible to setup a SpoofAction/SpoofCNAME action for a listening address?

[Remi Gacogne via dnsdist]

Hi Frank,

On 11/10/2023 07:01, F Even via dnsdist wrote:

Hello.  I was wondering if it was possible to do something like
setting up a netmask group for a QNameRule/SpoofAction, but instead of
tying it to requesting IP subnets, tie it to a specific set of
listening addresses.


If you want to match queries sent to a specific of listening addresses,
setting the 'src' parameter of 'NetmaskGroupRule'[1] to false should do 
what you want.


[1]: 
https://dnsdist.org/rules-actions.html?highlight=suffixmatch#NetmaskGroupRule

--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.7.5 and 1.8.2 released

[Remi Gacogne via dnsdist]

Hi,

Today we have released DNSdist 1.7.5 and 1.8.2, with absolutely no 
changes with, respectively, 1.7.4 and 1.8.1, apart from the fact that 
our DNSdist packages have been rebuilt against our own fork [1] of 
libh2o in order to mitigate CVE-2023-44487 [2], also known as HTTP/2 
rapid reset [3].


This attack exploits a vulnerability in most implementations of the 
HTTP/2 protocol, making it easier to cause a denial of service of HTTP/2 
servers by sending them crafted queries. While the vulnerability does 
not come from DNSdist's code, all versions of DNSdist supporting DNS 
over HTTPS are impacted by this issue if incoming DNS over HTTPS is 
enabled, which is not the case by default.


As we warned earlier, libh2o is no longer supported as a stable library, 
and there will be no official release fixing this issue. For this reason 
we have forked the official h2o repository and backported the fix to the 
2.2.x branch, making it available to the public. If you are not using 
our packages but are compiling DNSdist yourself, or relying on your 
distribution's packages, please ensure that you are using a patched 
version of libh2o in order to be protected.


In the very near future we will be releasing DNSdist 1.9.0 where DNS 
over HTTPS is provided by the nghttp2 library, so we do not have to rely 
on h2o any longer.


Please see the DNSdist website [4] for the current documentation.

Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [5].


The tarballs (1.7.5 [6], 1.8.2 [7]) and theirs signatures (1.7.5 [8], 
1.8.2 [9]) are available on the downloads website, and packages for 
several distributions are available from our repository [10].


Docker images have not been updated yet but will be soon.

[1]: https://github.com/PowerDNS/h2o/tree/v2.2.6%2Bpdns
[2]: https://www.cve.org/CVERecord?id=CVE-2023-44487
[3]: 
https://blog.cloudflare.com/technical-breakdown-http2-rapid-reset-ddos-attack/

[4]: https://dnsdist.org
[5]: https://github.com/PowerDNS/pdns/issues/new/choose
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.7.5.tar.bz2
[7]:
https://downloads.powerdns.com/releases/dnsdist-1.8.2.tar.bz2
[8]:
https://downloads.powerdns.com/releases/dnsdist-1.7.5.tar.bz2.sig
[9]:
https://downloads.powerdns.com/releases/dnsdist-1.8.2.tar.bz2.sig
[10]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist 1.7.4 Debian Bullseye vs 1.8.4 Bullseye

[Remi Gacogne via dnsdist]

Hi!

On 05/10/2023 10:41, Aleš Rygl via dnsdist wrote:
     Thanks for your response. After some deep documentation reading and 
config tweaking I am nearly on the previous values regarding CPU load, 
apart from latency, which is still higher (1.3ms -> 2.3ms). I suspect a 
different way the latency is likely computed (I noticed a new set of 
latency counters for TLS, TCP, etc.) here.  The key configuration 
parameter is setMaxTCPClientThreads(). Changing anything else (cache 
shards, number of listeners, etc.) has nearly no impact. We had 256 with 
1.7.4. now it is 16. Going up here means a rapid increase of CPU load, 
having less than 16 means dropping TCP connections in showTCPStats(), 
where Queued hits Max Queued. Insane values like 1024 kills the CPU. We 
have a physical server with 16 phys. cores, OS sees 32 cores.


OK, this is clearly unexpected. I mean, since 1.4.0 you should not be 
needing more TCP worker threads than the number of cores, since a single 
worker can handle a lot (easily thousands) of TCP connections, but 
having a larger value should not kill the CPU so I'm wondering if we are 
busy-looping somewhere. I have not been able to reproduce that so far, 
so I would be really interested in seeing the perf output if you can get 
it.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] greqp() output columns

[Remi Gacogne via dnsdist]

On 02/10/2023 19:17, Christoph via dnsdist wrote:
I don't think we have a way to log only these, unfortunately :-/ If 
you have the dnsdist console set up, you can use grepq('1000ms') to 
look at all queries that took more than 1 second, which is usually 
indicative of a problem, or even grepq('2000ms'), as dnsdist records 
timeouts with a very high response time.


Thanks for this suggestion.

out of ~200 lines from the grepq('3000ms') output 184 lines end with
... T.O RD No Error. 0 answers

examples:
aPPLE.CoM. A T.O   RD    No Error. 0 answers
fACeboOK.COm.     T.O   RD    No Error. 0 answers

does "T.O" in the Lat. column stand for timeout?


Yes, it means that dnsdist believes it did not get a response from the 
backend in time.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/

___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] backend drops metrics for TCP

[Remi Gacogne via dnsdist]

Hi Christoph,

On 13/09/2023 07:30, Christoph via dnsdist wrote:

I've switched back to using UDP.
Is there an easy way to log queries that timeout (2s) - and not log any 
others? To investigate some examples further?


I don't think we have a way to log only these, unfortunately :-/ If you 
have the dnsdist console set up, you can use grepq('1000ms') to look at 
all queries that took more than 1 second, which is usually indicative of 
a problem, or even grepq('2000ms'), as dnsdist records timeouts with a 
very high response time.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist latency bucket metric still broken in 1.8.0?

[Remi Gacogne via dnsdist]

Hi!

On 03/09/2023 11:08, Christoph via dnsdist wrote:
latency-doh-avg100 contains only a single avg value compared to 
latency-bucket.
Was there a specific reason, for not having a latency-bucket for DoH/DoT 
queries as well?


I do not recall whether this was an explicit decision, but my guess is 
that we were not expecting as much scrutiny over the DoT/DoH latency as 
with the UDP one. I am very willing to add latency-bucket for DoT, DoH 
and the upcoming DoQ, so I have put the issue you opened into the 1.9 
milestone.


Thanks!
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist 1.7.4 Debian Bullseye vs 1.8.4 Bullseye

[Remi Gacogne via dnsdist]

Hi Ales,

On 25/09/2023 16:09, Aleš Rygl via dnsdist wrote:
    I would to kindly ask for help or and advice. I have just upgraded 
one of our dnsdist instances from 1.7.4 do 1.8.4 together with OS 
upgrade (Debian 11.7 to 12.1). Everything works fine, no issues 
observed apart some deprecated config references. What is a big 
surprise to me is CPU usage. The newer version has nearly two times 
higher CPU consumption in userspace. I am nearly at 80% CPU with 16 
physical cores (was about 40%). We have a lot of TLS (DoT) sessions 
(30k) and 60kqps in total (30k via DoT) here. The latency measured by 
dnsdist went up also. We are collecting all the metrics dnsdist 
produces via graphite so I can check counters, what could be wrong.


Wow, that's awful. It's the first time I hear about such a regression, 
and I really would like to understand what is going on.
1/ Are you using our packages, compiling yourself, or perhaps using the 
Debian ones?
2/ Do you think it would be possible for you to try downgrading the 
instance to 1.7.4 on Debian 12.1? It might help us pinpointing whether 
the issue is related to a system change (I have seen people complain 
about the performance of OpenSSL 3.0.x compared to 1.1.1x, for example).

3/ Would you mind sharing your configuration?
4/ And finally, do you think it would be possible for you to collect a 
perf trace on this instance? It would require installing linux-perf, if 
possible the debug symbols for dnsdist (dnsdist-dbgsym) then running 
'perf record --call-graph dwarf -p  -o 
' for a few dozens of seconds to collect a trace, 
stopping it with Ctrl+C and finally getting a report with "perf report 
-i  --stdio". It should tell us where the CPU 
usage is going.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Does dnsdist try the next server inside a pool, when the first does not answer?

[Remi Gacogne via dnsdist]

Hi Tobias,

On 25/09/2023 06:18, Schnurrenberger Tobias (ID) via dnsdist wrote:

We are using multiple resolvers in the same pool and we set the 
setServFailWhenNoServer option. There is also an overflow configured, which 
allows only 1 qps to this pool.

What happens when the first server in the pool does not answer the query within 
the configured setUDPTimeout?
Is the same query sent to the next server inside the pool?


No, it does not.


Or does dnsdist reply to the client with SERVFAIL without trying another server?


In the exact case of the selected backend timing out, dnsdist will not 
reply to the client at all. setServFailWhenNoServer() controls what 
happens when all servers in the selected pool are down, but not when a 
server was considered to be available but did not answer in time.



Does dnsdist only switch to the next server, if the state of the first one is 
'down'?


In your case, yes. With the whashed load-balancing policy that you are 
using, dnsdist wil selected a backend among the ones that are considered 
available, based on the latest health-check attempts, using a hash of 
the queried name.
The reasoning behind this behaviour was that most applications/stub 
resolvers will retry quite quickly over UDP, often before 2 seconds 
which is the default value of setUDPTimeout, and thus it does not make 
sense to increase the load on the backend.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.9.0-alpha1 released

[Remi Gacogne via dnsdist]

Hello!

We are very happy to be releasing the first alpha release of what will 
become DNSdist 1.9.0!


The most important change since 1.8.1 is that incoming DNS over HTTPS 
requests are now handled by the nghttp2 library, instead of the h2o one. 
This change should be transparent for most users, since we made sure to 
preserve the existing features and configuration directives. Switching 
to nghttp2 allows us to support hardware acceleration for TLS exchanges, 
using for example Linux's kTLS or Intel Quick-Assist Technology. It also 
reduces our footprint on low-end devices by not requiring an additional 
library, since nghttp2 was already used for outgoing DNS over HTTPS 
requests. Finally, while it was a long time coming, h2o is officially 
[1] no longer maintained in a way that makes it possible to use it as a 
stable library. Technically it will still be possible to revert to the 
use of h2o for incoming DNS over HTTPS in DNSdist 1.9.x, but we will 
remove that support after that.


We have made a lot of small improvements since 1.8.x as well, like 
adding Lua bindings to access selectors and actions, more fields of a 
DNS header in Lua actions, and adding metrics for health-check events.


We still have several wonderful features planned for 1.9.0 that have not 
been merged yet, but rest assured that the final release will not be boring!


Packagers need to be aware that SNMP support is no longer enabled by 
default, as it had been causing integration issues in some environments 
for a while, but it's still enabled in our packages.


Speaking of packages, we are now publishing SLSA attestations along with 
our packages, making it possible to verify how exactly they were built 
and reproduce our workflow, providing strong guarantees against 
supply-chain attacks. Please get in touch if you want to know more!


Please see the DNSdist website [2] for the more complete changelog [3] 
and the current documentation. The upgrade guide is also available there 
[4].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [5].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [6] and its signature [7] are available on the 
downloads website, and packages for several distributions are available 
from our repository [8].


[1]: https://github.com/h2o/h2o/issues/3230
[2]: https://dnsdist.org
[3]: https://dnsdist.org/changelog.html#change-1.9.0-alpha1
[4]: https://dnsdist.org/upgrade_guide.html#x-to-1-9-0-alpha1
[5]: https://github.com/PowerDNS/pdns/issues/new/choose
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-alpha1.tar.bz2
[7]:
https://downloads.powerdns.com/releases/dnsdist-1.9.0-alpha1.tar.bz2.sig
[8]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] backend drops metrics for TCP

[Remi Gacogne via dnsdist]

Hello!

On 11/09/2023 22:34, Christoph via dnsdist wrote:

when playing around with things to reduce the drop rate I noticed
that TCP based backends always have 0 drops in showServers() output and 
these metrics:

dnsdist_server_drops
dnsdist_downstream_timeouts

Is that always the case and that counter has no meaning for TCP based 
backends or can this counter be non-zero for TCP backends as well?


This counter will always be 0 for TCP backends indeed, it is only 
incremented when we give up waiting on a UDP response.



dnsdist's CPU usage doubled after switching to TCP via tcpOnly=true
and the DNS timeout rate as measured by the application generating the 
queries running on the same host as dnsdist actually increased after 
switching dnsdist to use TCP instead of UDP. So switching to TCP 
eliminated the drops problem when measured by dnsdist but it made things 
worse for the application.


The default timeout for TCP backends is set at 30s, while for UDP 
responses it is at 2s. So it is very possible that dnsdist no longer 
considers the response a timeout but the application now does. You might 
try to tune the 'tcpRecvTimeout' on `newServer`. Note that this suggests 
that the backend is slow to answer, so tuning dnsdist might not help at 
all and investigating why the backend struggles with these queries might 
be needed.



All of these values are also at 0:

dnsdist_server_tcpdiedsendingquery{address="127.0.0.1:54"} 0
dnsdist_server_tcpdiedreadingresponse{address="127.0.0.1:54"} 0
dnsdist_server_tcpgaveup{address="127.0.0.1:54"} 0
dnsdist_server_tcpreadtimeouts{address="127.0.0.1:54"} 0
dnsdist_server_tcpwritetimeouts{address="127.0.0.1:54"} 0
dnsdist_server_tcpconnecttimeouts{address="127.0.0.1:54"} 0


These are indeed the ones that would indicate a problem between dnsdist 
and a TCP backend, as seen by dnsdist.



Since sockets=NUM in newServer() is only for UDP and
dnsdist_server_tcpcurrentconnections{address="127.0.0.1:54"} 10
suggests it uses only 10 TCP sockets. How can this be configured?
sockets was set to 32, so this implicit change when sitching from UDP to 
TCP might also have an effect here.


dnsdist will create as many outgoing TCP connections as needed by 
default, unless instructed otherwise via 'maxConcurrentTCPConnections' 
on `newServer`. So from dnsdist's point of view there was no need for 
more TCP connections, apparently. Do you enable out-of-order processing, 
via 'maxInFlight' on `newServer`? If so, are you sure that the backend 
actually supports it?


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Dnsdist as high availability mode

[Remi Gacogne via dnsdist]

Hi!

On 10/09/2023 16:31, Affan Basalamah via dnsdist wrote:
I want to know if it’s  possible to setup  dnsdist as two server and 
configured with first hop redundancy protocol such as VRRP (Keepalived 
on Linux, CARP on FreeBSD), so dnsdist can operate as highly available 
as possible, at least in fail over mode.


If it’s possible, I haven’t seen the examples on the official dnsdist 
documentation. Can someone point me to any document or other references?


Yes, I know of a few setups that are using dnsdist with Keepalived. 
There is nothing particular to do on dnsdist's side to make it work, 
which is why it's not mentioned in the documentation.
We will of course accept pull requests updating the documentation if you 
find out that some parts could use some words, though :)


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] PowerDNS DNSdist 1.8.1 released

[Remi Gacogne via dnsdist]

Hello!

We are very happy to release DNSdist 1.8.1 today, a maintenance release 
fixing a few bugs reported since 1.8.0:


- Several bugs have been fixed in the health-check code, including one 
issue that could have resulted in some health-check responses to be lost
- A crash has been fixed when dealing with DNS over HTTPS queries for 
which a X-Forwarded-For header overrides the initial source IP, which is 
not enabled by default
- Re-connection failures are now more carefully handled for backend UDP 
sockets, avoiding high CPU usage in some network topology changes

- Self-answered UDP responses with recvmmsg are not properly accounted for
- A memory leak when processing TLS tickets with OpenSSL 3.x has been fixed
- Cache hit and miss metrics with DoH queries are now properly accounted for
- Christof Chen fixed an issue with SpoofAction, by copying the QClass 
from the request
- A race has been fixed when creating the first TLS connections to a 
backend, which could have led to sub-optimal TLS session reuse

- Short reads are now properly handled when doing backend upgrade discovery
- Winfried Angele fixed an accidental change of disableZeroScope to 
disableZeroScoping
- The group ownership of the dnsdist.conf file is now properly set when 
installed via RPM
- Houtworm fixed the webserver configuration template for our docker 
container

- phonedph1 fixed the console description of PoolAction and QPSPoolAction

In addition to these fixes, Jacob Bunk made the TSIG query type 
available from Lua, and we improved the accounting of eBPF dynamic blocks.


Finally it is now possible to declare custom metrics at runtime for use 
from Lua, and Lua FFI inspection functions are automatically loaded at 
runtime.


Please see the DNSdist website [1] for the more complete changelog [2] 
and the current documentation. The upgrade guide is also available there 
[3].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.8.1
|3]: https://dnsdist.org/upgrade_guide.html#x-to-1-8-0
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.8.1.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.8.1.tar.bz2.sig
[7]: https://repo.powerdns.com

--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Matching corrupt DNS queries?

[Remi Gacogne via dnsdist]

Hi Jacob,

On 13/08/2023 13:07, Jacob Bunk Nielsen via dnsdist wrote:

We are sometimes seeing UDP DNS queries that come in with the TC flag
set to true. That doesn't make sense to send such queries as the client
should of course just make that query over TCP.

But how do I match those queries in dnsdist? The DNSHeader class has a
:setTC() function, but not a :getTC() function.


Right, it was indeed missing. [1] adds it, and will likely be backported 
to 1.8.x.



Also, it would be great
to have a generic way to match on header flags like you can do on e.g.
query types.

Something like:

HeaderRule(DNSFlags.TC, true)

or similar, but I don't find anything like this in the docs.


I don't think we have such a rule yet, and I would gladly add it to 
dnsdist. Would you mind opening a feature request so it doesn't get 
forgotten?


[1]: https://github.com/PowerDNS/pdns/pull/13135

Cheers,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature.asc
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist 1.8.0 thread spinning

[Remi Gacogne via dnsdist]

On 15/07/2023 09:42, Otto Moerbeek via dnsdist wrote:

This is likely https://github.com/PowerDNS/pdns/pull/12726

ATM this is not marked for backporting to 1.8.x. Don't know if that is
an omission.


It was, I added the 'backport to dnsdist-1.8.x' flag in the meantime. 
Thanks!


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Exclude domains with dynBlockRulesGroup ?

[Remi Gacogne via dnsdist]

Hi Denis,

On 12/07/2023 12:24, Denis MACHARD via dnsdist wrote:
How to exclude some domains with the dynamic block feature 
(dynBlockRulesGroup),
Is it possible ? The documentation is not clear on this, if anyone has 
an example.


We should document this more clearly, there are two types of rules in a 
DBRG:


- setQueryRate, setRCodeRate, setRCodeRatio, setQTypeRate and 
setResponseByteRate work by looking at the queries and responses present 
in the ring buffers grouped by client IP, so they can decide to apply an 
action on a given IP. Therefore the excludeRange, includeRange 
directives apply to these rules to allowlist and denylist some IPs/ranges.


- setSuffixMatchRule and setSuffixMatchRuleFFI work by looking at the 
responses present in the ring buffers grouped by subdomains, so they can 
decide to apply an action on a given domain or subdomain. Therefore the 
excludeDomains directive apply to these rules to prevent a domain and 
its children from being blocked.


So you cannot exclude an IP or a range from 
setSuffixMatchRule/setSuffixMatchRuleFFI, and neither can you exclude a 
domain from the other rules.


I hope that helps!

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Re: Some statistics I would like to have

[Remi Gacogne via dnsdist]

On 13/06/2023 10:34, Stephane Bortzmeyer wrote:

Ah, yes, thanks. But the numbers are disconcerting. I see twice as
much queries per connection with IPv4 than IPv6, but only for DoT. For
DoH, it is the opposite. This may be simply because it is a small
resolver so the numbers are not statistically significant.


Interesting, it would be nice to know if large resolvers are also seeing 
very different numbers. I'll ask around.



But more importantly, dnsdist forwards queries received over DoH using UDP,
while queries received over DoT are forwarded over TCP, unless the backend
has been configured as TCP-only, DoT or DoH.


Which is the case, no?

newServer({address="[::1]:853", name="Local-Unbound", tls="openssl", 
subjectName="dot-backend.bortzmeyer.fr", validateCertificates=true, useClientSubnet=false, maxInFlight=1024})


Indeed, so queries received over DoT and DoH are forwarded in the same 
way. I wonder why the latency numbers are so different. Perhaps we are 
seeing different query patterns between DoT and DoH clients?


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Some statistics I would like to have

[Remi Gacogne via dnsdist]

Hi Stéphane,

On 12/06/2023 14:44, Stephane Bortzmeyer via dnsdist wrote:

I'm wondering about the average number of DNS queries per DoT/DoH
connection (to see if the setup "cost" is amortized over enough
requests). I do not find something like that in the output of
dumpStats (I have the number of queries but not the number of
connections). Any idea?


You should be able to find what you are looking for in the output of 
"showTCPStats()".



Also, the Web interface tells me:
Average response time: [...] DoT 357.45 ms, DoH 531.12 ms
I assumed, may be wrongly, that this response time depended on the
time required to fetch the response and so was the same for DoT and
DoH. Why not?


The response time is computed from the moment we received the query from 
the client until after the response has been sent, so while this is 
usually indeed pretty much equal to the time required to fetch the 
response, it also records the time needed for internal processing, there 
might be some differences between protocols.
But more importantly, dnsdist forwards queries received over DoH using 
UDP, while queries received over DoT are forwarded over TCP, unless the 
backend has been configured as TCP-only, DoT or DoH. So it might be why 
you are seeing a difference, although this usually tends to be the other 
way around with DoH being faster than DoT because DNS servers have 
historically been optimized for UDP.


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist restrict udp source port

[Remi Gacogne via dnsdist]

Hi Mahdi,

On 11/05/2023 10:47, Mahdi Adnan via dnsdist wrote:
  Is it possible to restrict dnsdist to a certain port range, something 
similar to what pdns have "udp-source-port-min, udp-source-port-max, 
or udp-source-port-avoid"? I couldn't find such an option on the man 
page or in the documentation.


I'm afraid there is no such option, dnsdist is relying on the operating 
system to get random source ports.
In theory you could set a source port in addition to an address in the 
'source' parameter of "newServer" when defining a backend and then only 
this port will be used to reach the backend, but this will not not work 
if you want to use more than one source port using the 'sockets' 
parameter, and more importantly will prevent dnsdist from opening more 
than one TCP connection to the backend which is very likely not what you 
want.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist latency bucket metric still broken in 1.8.0?

[Remi Gacogne via dnsdist]

Hi Christoph,

On 14/04/2023 17:04, Christoph via dnsdist wrote:

here is our dnsdist.conf,
maybe it helps to reproduce the issue.


If I'm not mistaken, you are mostly dealing with DoT and DoH queries, 
not UDP ones?
I'm asking because since 1.8 these latency metrics are now only updated 
for UDP queries, and we have new ones that are broken down by protocol, 
as implemented in [1]. We have more information at [2] and a sample 
prometheus output at [3].
We do know it will require updating a few dashboards, but exactly how 
the latency was measured when several protocols were in use has been 
very hard to understand since dnsdist 1.0, with some advocating for the 
UDP latency to be only one meaningful and others wanting a global 
overview regardless of the protocol. Both use-cases can now be done by 
aggregating the metrics as needed.


Note that I still don't understand how we can end up with all recorded 
queries in the "slow" bucket ("dnsdist_latency_bucket{le="+Inf"}") and 
still get 0 as the sum ("dnsdist_latency_sum"), but I have not been able 
to reproduce this behaviour so I don't know what is going on.


[1]: https://github.com/PowerDNS/pdns/pull/11716
[2]: 
https://dnsdist.org/statistics.html?highlight=latency#latency-doh-avg100
[3]: 
https://dnsdist.org/guides/webserver.html?highlight=latency#get--metrics


Hope that helps,

Remi



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist latency bucket metric still broken in 1.8.0?

[Remi Gacogne via dnsdist]

Hi Christoph,

On 13/04/2023 19:55, Christoph via dnsdist wrote:

ever since [1] got the  dnsdist-1.8.0 milestone
we were looking forward to the 1.8.0 release
and were also a bit surprised that this regression
will not be in a 1.7.x bugfix release.


The fix not being backported is an oversight, I added the "backport to 
1.7.x" flag so we include it in an upcoming 1.7.x release.



Recently we upgraded our dnsdist instances to 1.8.0
but the upgrade did not improve the values in dnsdist_latency_bucket.
Now after the upgrade, the graph show basically a flat line.
This only affects our FreeBSD servers, not our Debian based dnsdist 
instances.


That's weird. Would you be able to share the prometheus output, or the 
dumpStats() one, so we know if this is the same bug or a related one?


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist 1.7 : allow only A request

[Remi Gacogne via dnsdist]

Hi,

On 17/03/2023 16:23, david n via dnsdist wrote:

BUT : for the "any" request I have this result, and I don't know if it can 
produce something bad, have you any idea ?

[root@node ~]# dig any www.toto.com @X.X.X.X
;; communications error to X.X.X.X#53: end of file
;; communications error to X.X.X.X#53: end of file


dig defaults to sending ANY queries over TCP, instead of UDP, and here 
dnsdist closes the TCP connection as soon as the query is dropped, 
triggering the message you are seeing. It's not clear to me why the 
connection is dropped from your configuration, though, have you shared 
the actual configuration?


dnsdist could keep the TCP connection around instead but it would be too 
nice, because timeouts over TCP are usually quite longer than over UDP 
and the connection would just sit there, consuming socket descriptors 
for nothing.


Hope that helps,

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] Third Release Candidate of PowerDNS DNSdist 1.8.0

[Remi Gacogne via dnsdist]

Hello!

We are very happy to release the third candidate of what will become 
dnsdist 1.8.0!


This release contains fixes for several issues that were found in the 
second release candidate.


- #12641: Use the correct source address when harvesting failed
- #12639: Fix a race when a cross-protocol query triggers an IO error
- #12638: Report the TCP latency for TCP-only Do53, DoT and DoH backends
- #12648: Report per-incoming transport latencies in the web interface

The first one is actually a follow-up to the "dnsdist is responding from 
the wrong source IP address in some setups" which was incompletely 
corrected in rc2. The second one is a race condition that might have 
been occurring in very specific cases of network errors during 
asynchronous processing of cross-protocol queries. The last issue is a 
bit more complicated: in 1.8.0 we decided to break down the latency 
metrics to provide a more accurate view of what was actually going on:


- global latency metrics are now per incoming protocol (Do53 UDP, Do53 
TCP, DoT, DoH)
- backend latency metrics are split between UDP (Do53) and TCP (Do53 
TCP, DoT, DoH)


This change brought some adjustment in the interfaces consuming these 
metrics, and it was reported that this was not quite right. The web 
interface, for example, was only reporting the UDP-based metrics and not 
the other ones.


Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation. The upgrade guide is also available there 
[3].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.8.0-rc3
|3]: https://dnsdist.org/upgrade_guide.html#x-to-1-8-0
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.8.0-rc3.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.8.0-rc3.tar.bz2.sig
[7]: https://repo.powerdns.com


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] Second Release Candidate of PowerDNS DNSdist 1.8.0

[Remi Gacogne via dnsdist]

Hi!

We are very happy to release the second candidate of what will become 
dnsdist 1.8.0!


This release contains fixes for a few issues that were found in the 
first release candidate, the most important one being that dnsdist was 
responding from the wrong source IP address in some setups, which was 
reported by multiple users. Many thanks to them!


- #12586: Fix the harvesting of destination addresses, so we reply from 
the correct source IP in all cases

- #12587: Skip signal-unsafe logging when we are about to exit, with TSAN
- #12588: Fix compilation with DoH disabled (Adam Majer)
- #12589: YaHTTP: Better detection of whether C++11 features are available
- #12592: Only increment the ‘servfail-responses’ metric on backend 
responses (phonedph1)

- #12593: Clean up the fortify and LTO m4 by not directly editing flags
- #12615: Add Lua bindings for PB requestorID, deviceName and deviceID

Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation. The upgrade guide is also available there 
[3].


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


We are immensely grateful to the PowerDNS community for the reporting of 
bugs, issues, feature requests, and especially to the submitters of 
fixes and implementations of features.


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.8.0-rc2
|3]: https://dnsdist.org/upgrade_guide.html#x-to-1-8-0
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-8.0-rc2.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.8.0-rc2.tar.bz2.sig
[7]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Define from which source dnsdist is sending a reply

[Remi Gacogne via dnsdist]

Hi Sandro,

On 03/03/2023 16:28, Sandro Bolliger via dnsdist wrote:

Is it possible to send a reply from a specific IP as source in dnsdist? I use 
multiple different IPs on the Loopback interface of my dnsdist machine. The IPs 
are routed to that server. Now dnsdist is replying with the interface IP on 
which it has connectivity to the other machines. This is causing trouble with 
as an example Ubuntu systems that now refuse to use the reply from the wrong 
source IP. As an example Ubuntu sends its DNS query to 192.0.2.1 and gets a 
reply from 203.0.113.2. It then tells me ;; reply from unexpected source: 
203.0.113.2#53, expected 192.0.2.1#53.

Best case would be to send the reply from the same IP I received the request on.


Yes, if dnsdist does not do that it's a bug. Are you by any chance 
running 1.8.0-rc1, which has a known issue that is very similar to that 
(see [1], and the fix in [2])? If so, we will release -rc2 next week.


[1]: https://github.com/PowerDNS/pdns/issues/12581
[2]: https://github.com/PowerDNS/pdns/pull/12586
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] First release candidate of dnsdist 1.8.0

[Remi Gacogne via dnsdist]

Hi Vincent,

On 01/03/2023 11:42, Vincent Schönau via dnsdist wrote:

As of 1.7, dnsdist would respond from the IP address it received the query on 
when configured to listen on multiple IP addresses with addLocal(). My install 
of 1.8 (from the Ubuntu repo, 1.8.0~rc1+master.35.g8243d1786-1pdns.jammy) does 
not appear to do that, it returns a response from a different IP (albeit in the 
same network) instead:

Is this intended behavior?


Not at all, it completely is a bug. It was reported a few days ago [1] 
and is fixed in git [2]. We are waiting a bit to see if other issues are 
reported and will issue a rc2 somewhere next week.


Thanks!

[1]: https://github.com/PowerDNS/pdns/issues/12581
[2]: https://github.com/PowerDNS/pdns/pull/12586
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] First release candidate of dnsdist 1.8.0

[Remi Gacogne via dnsdist]

Hello!

We are very happy to release the first candidate of what will become 
dnsdist 1.8.0!


This release contains a significant amount of changes since the last 
major release, 1.7.0, which was released a bit over a year ago. We try 
to stick to a major release every six months, but this one took a bit 
longer than expected as we tackled a few challenges:


Low-end devices friendly


We know, based on the feedback we get from the users that interact with 
us, that dnsdist is used in a lot of different environments, from very 
large installations dealing with millions of queries per second to very 
small computers running in a closet somewhere! While we have until now 
been more focused on the first case, we have been getting a lot of 
interest coming from the very-low end of the spectrum: low-end devices, 
like customer premises equipment (CPEs), with very few resources. We 
realized that while other open-source components do a good job of 
providing traditional DNS services in that world, there is a need for 
software providing DNS over TLS and DNS over HTTPS support, to protect 
the confidentiality and integrity in the first mile of the internet access.


We knew that dnsdist was already successfully used on small devices, 
like raspberry pis, and that our memory and CPU usage was quite low, so 
we were surprised to learn that people were struggling to meet the very 
stringent requirements of some devices, and decided to have a look. This 
was a very interesting journey into flash-based filesystems of a few 
dozen megabytes, proportional set size memory usage, and low-powered CPUs.


Long story short, we managed to drastically reduce our memory usage and 
our CPU consumption, especially with very low QPS rates. We developed a 
new way of doing health-checking for these environments, only doing an 
actual active health-check after detecting failures from normal traffic. 
We also introduced a few options to reduce our binary size where it 
matters, like on OpenWrt builds.


OpenWrt integration
---

We wrote the necessary code to make dnsdist play nicely with OpenWrt's 
native configuration format, Unified Configuration Interface (UCI), so 
that it is easy to set up dnsdist via the usual interfaces, including 
the Web UI.


We also provide DHCP integration, so that dnsdist can learn about 
devices on the local network and provide native DNS resolution for these 
devices.


This integration is not yet merged into the OpenWrt tree as it requires 
some features that will only be available once 1.8.0 final has been 
released. Stay tuned, or reach out if you want a quick peak!


Hostile networks


We also realized that we could no longer rely on the network path 
between dnsdist and its backend to be trusted: while this is true when 
dnsdist is deployed on the same box, rack or datacenter as the backend, 
this no longer is when it is deployed on a CPE and instructed to forward 
its queries to a remote recursive resolver like Quad9.


Of course we strongly advise using DNS over TLS and/or DNS over HTTPS to 
secure that path, but this is unfortunately not always possible. We 
learned the hard way that in some countries ISPs are not only providing 
DNS over plain UDP only, without even supporting plain TCP, they are 
also still blocking attempts to connect to an external resolver via a 
more secure channel.


To work around that issue, we implemented new features to make dnsdist 
suitable as a proxy with an untrusted network path to the resolver, 
using well-known methods: random ports and random IDs. These are not 
enabled by default because they come at a cost, which we don't want to 
impose when it is not necessary.


Discovery of Designated Resolvers
-

It's one thing to support DNS over TLS and DNS over HTTPS both inbound 
and outbound, but it really does not help if the client does not know 
that you do, or if the configuration does not tell dnsdist that the 
backend does.


The IETF has been working for quite some time now on a new mechanism 
that leverages the SVCB record type to actually advertise that a secure, 
encrypted endpoint is available for use: Discovery of Designated 
Resolvers (DDR) [1].


Since 1.7.0 dnsdist has been able to advertise DoT and DoH support to 
the client via SVCB records, but that requires writing a few lines of 
Lua to configure it. In 1.8.0, we have integrated that process into the 
OpenWrt configuration, requiring a single click to enable DDR 
advertisement to all the local clients, allowing Android and iOS devices 
to automatically upgrade to a secure channel.


We also taught dnsdist how to use DDR to detect whether a given backend 
can be upgraded from plain Do53 to DoT and DoH, so that we switch to a 
secure channel as soon as it becomes available, and fallback to Do53 if 
needed.


Faster TLS
--

To be able to keep pushing for broader adoption of DoT and DoH, it is 

Re: [dnsdist] [EXT] Re: Performance/tuning sanity check

[Remi Gacogne via dnsdist]

On 03/01/2023 17:07, Dan McCombs wrote:
I spent some time looking through the thread Nicolas shared at 
https://mailman.powerdns.com/pipermail/dnsdist/2017-April/000281.html 
 
of his performance tuning experience. Do you know if it's still the case 
in newer versions of dnsdist to need multiple instances of dnsdist 
running on a machine this size (32 cores) rather than just additional 
listeners via addLocal to really push past 400kQPS? I just want to make 
sure I'm not working off out-of-date information.


A lot has changed with regard to scalability on machines with a great 
number of cores. I remember us doing a lot of work in that area in 
1.3.0, and we also added a few more improvements since then (false 
sharing work-around, more sharding, more efficient locking, etc)


I have personally reached quite more than 400k QPS with a single dnsdist 
process during my tests, but you might be able to get better performance 
with several processes as we are still sharing some state between 
threads. For example Otto has been working on more scalable counters in 
the recursor lately, and this is something that I would like to 
implement in dnsdist. It will likely have to wait until 1.9, so around 
Q3 this year, unless someone beats me to it :)


Please report back with your findings, as it is really helpful to hear 
about different setups than the ones we already do our performance 
testing on.


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Performance/tuning sanity check

[Remi Gacogne via dnsdist]

Hi Dan, Klaus,

I just noticed I messed up with my previous response and it never made 
it to the list, so I'm re-posting it now.


I don't see anything wrong with your configuration, Dan. You could look 
at the metrics to see if you are experiencing contention in the 
packet-cache and ring-buffers, and increase the number of shards in that 
case.


Other than that, I agree with Klaus that tuning the system itself 
becomes necessary above 400k QPS.

In addition to IRQ balance, I would suggest looking at:
- increasing receive and send buffers: see [1]
- unloading any module related to connection tracking
- setting a proper UDP flow load-balancing policy, see 'ethtool 
rx-flow-hash udp4 sdfn' and the likes
- setting up CPU pinning [2] so that 1/ all dnsdist threads are on the 
same NUMA cores 2/ dnsdist receivers threads are on the CPU cores 
handling NIC receive queues (which might involve disabling IRQ balance 
and directly assigning the IRQ affinity for the NIC queues)


Hope that helps,

[1]: https://dnsdist.org/reference/tuning.html#setUDPSocketBufferSizes
[2]: see the cpus option on 
https://dnsdist.org/reference/config.html#addLocal


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Handling lack of caching of TC responses

[Remi Gacogne via dnsdist]

Hi Dan,

On 18/11/2022 02:51, Dan McCombs via dnsdist wrote:
Is that something that's expected to happen once the full response has 
been returned from a downstream over TCP? Is there some way to force TC 
responses to have at least some minimal TTL? Or some way to have dnsdist 
use its cached response rather than going back to the downstream in that 
case on UDP queries?


This is indeed an unfortunate consequence of the fact that there is no 
TTL on a truncated answer, and no way to set one, so the cache does not 
even try to store it.
Perhaps it might make sense to cache these TC=1 answers for a 
configurable, very short amount of time. We already have a setting to 
cache "temporary failures", meaning Server Failure and Refused 
responses, for a short amount of time, so it would not be hard to 
implement the same behaviour for truncated answers.
I think we have been reluctant to do that until now because some 
backends actually send TC=1 answers to specific clients, like in the RRL 
Slip case, and do not expect these answers to be served to other 
clients, but that might be OK as long as the caching is configurable and 
can be disabled.


Would you mind opening a feature request on GH [1] so we can track this? 
I would also happily merge a corresponding pull request, of course :)


Thanks!

[1]: https://github.com/PowerDNS/pdns/issues/new/choose

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Responding from cache when all pool servers are down?

[Remi Gacogne via dnsdist]

Hi Aaron,

On 01/11/2022 23:03, Aaron de Bruyn via dnsdist wrote:
Is there a way to specifically respond to cached items and return 
SERVFAIL if it's not in the cache?


I'm afraid there is no way to do exactly that. I guess we would need a 
selector with the ability to look into the packet-cache (so you could 
use RCodeAction only if the entry is not there) or a chain of rules that 
is executed after the packet-cache lookup.


But if I understand correctly what you are looking for, 
setServFailWhenNoServer(true) [1] should do what you want:


- look into the packet cache and return the result on a hit
- forward the query to an available server in the selected pool, if any
- return a ServFail response if there is no available server in the 
selected pool


[1]: 
https://dnsdist.org/guides/serverselection.html?#setServFailWhenNoServer


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Client query id in the dq-object?

[Remi Gacogne via dnsdist]

Hi Tom,

On 04/11/2022 08:02, Tom via dnsdist wrote:
So the idea was to set a dnsdist rule on which I can set the 
AllowedDebugSRC (the admin's IP, to prevent, that anybody else could 
trigger the debug), check the QueryID and then send this kind of 
requests to a debug-enabled BIND, which then write a debug log from the 
received query. Someting like this:



AllowedDebugSRC = newNMG()
AllowedDebugSRC:addMask("1.2.3.4/32")
function qidlog(dq)
     if(AllowedDebugSRC:match(dq.remoteaddr) and  == 0) then
     print("Debugging from " .. dq.remoteaddr:toString() .. 
" with query id" .. )

     return DNSAction.Pool, "bind-querylog"
     end
end
addAction(AllRule(), LuaAction(qidlog))


I'm afraid there is currently no way in dnsdist to influence the query 
ID in outgoing queries. This would not be easy to implement as it 
touches the way dnsdist matches the responses received to the backend 
against the outstanding queries, so that logic is quite deep in the core 
code of dnsdist.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] expungeByName from ALL pools

[Remi Gacogne via dnsdist]

Hi Christof,

On 05/10/2022 12:06, Chen, Christof via dnsdist wrote:
I use separate caches for various pools. Now I want to delete a given 
FQDN from the cache, but I don't know immediately in which cache it is - 
so I would like to simpy expungeByName it from all caches.


Is there a straightforward way to address all pools or iterate over all 
pools? I would like to avoid hardcoding my list of pools here:


getPool("poolA"):getCache():expungeByName(newDNSName("what.ever.name 
"))
getPool("poolB"):getCache():expungeByName(newDNSName("what.ever.name 
"))

[...several more pools with caches...]

Maybe some Lua magic with getPools()?


I'm afraid I don't see an easy way to do that, short of parsing the 
output of 'showPools()'. It would not be too hard to write a Lua 
function returning the list of all the existing pools, though. Would you 
mind opening a feature request on GH so that idea doesn't get lost?


Cheers,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Backend Questions

[Remi Gacogne via dnsdist]

Hello Klaus,

On 07/10/2022 10:53, Klaus Darilion via dnsdist wrote:

We use dnsdist with 1 single backend server (PDNS). So if this backend 
is overloaded, dnsdist will detect the backend as DOWN. Hence, the only 
server for this backend pool down. How will dnsdist behave if all 
servers for a backend pool are down? Will it stop senden queries to the 
backend, or will it still send queries to the DOWN server as there is no 
UP server available?


All of the built-in load balancing policies will stop forwarding queries 
when all the servers in the selected pool are down. It would be possible 
to write a custom load-balancing in Lua that does not do that, of 
course, but I don't think that's what you want in that case.


So it may be useful to disable healthchecks completely. How can this be 
done?


You can force a server in the "up" state using the 'setUp()' method, see 
[1].


My current config is a few years old tested with dnsdist 1.3. These 
days, dnsdist was faster when I added the listen port multiple times, 
and also add the single backend server multiple times, to have more 
receiver threads. For example:


Is it still (dnsdist 1.6/1.7) useful/necessary to add listenSockets and 
Backendserver multiple times to improve performance?


I'm afraid so, yes. We have been thinking about a better syntax/design 
for spawning multiple threads without having to duplicate the lines, but 
haven't settled on and implemented a new syntax yet. Ideally we would 
like to avoid breaking compatibility with existing setups, both for the 
configuration files and for the metrics, but we might have to make a 
breaking change in a new major version.


[1]: https://dnsdist.org/quickstart.html#changing-server-settings

--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Whitelisting IP addresses with XDP filtering

[Remi Gacogne via dnsdist]

Hi,

On 05/10/2022 09:30, Pierre Grié via dnsdist wrote:
In the meantime you could exclude the range using [1] to make sure that 
this is really the root cause of your issue.


We already identified that dnsdist was the root cause by restarting 
dnsdist after it inserted the IP in the DynBlock and checking it was 
truncating new queries event after whitelisting. This lead to the BPF 
map remaining unchagned (the IP was still in it, so queries were 
supposed to be TC but were whitelisted), and the new queries were not 
truncated anymore, as the DynBlock was empty on userspace side.


Great.

We might be able to get rid of that now, or at the very least we should 
make it optional.


That would really be a time-saver for us !


I opened a feature request ticket to track this at [1]. I tentatively 
set the milestone to 1.8.0 but I'm not sure I will have the time to look 
into this quickly.
If you, or someone else, wants to tackle it and open a pull request I 
think the second option I listed in the ticket should be fairly 
straight-forward to implement.


[1]: https://github.com/PowerDNS/pdns/issues/12061

Cheers,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Whitelisting IP addresses with XDP filtering

[Remi Gacogne via dnsdist]

Hi Pierre,

On 04/10/2022 17:59, Pierre Grié via dnsdist wrote:
I am currently working on a XDP BPF filter to work with dnsdist BPF maps 
which put the TC bit on packet from incoming IPs flagged by dnsdist, and 
I am trying to implement a whitelist system with an additional map that 
would contain IPs we would like to "whitelist" (i.e which would be 
allowed to perform UDP queries even when flagged by dnsdist and put in 
the BPF map with the DNSAction.Truncate action).


Sounds great!

The whitelisting mechanism work fine by itself, but it seems that when 
the whitelisted UDP query hits dnsdist after passing through the XDP 
filter, it is resend with the TC bit, thus forcing the client the retry 
with TCP. Is the DNSAction also enforced in userspace ?


Yes, the current behaviour is to add the rule to the userspace dynamic 
block even when eBPF filtering is enabled. It was initially done to 
prevent the dynamic blocks being bypassed on some distributions where 
the kernel was pretending that eBPF was working even though it was not.
We might be able to get rid of that now, or at the very least we should 
make it optional.
In the meantime you could exclude the range using [1] to make sure that 
this is really the root cause of your issue.


[1]: 
https://dnsdist.org/reference/config.html#DynBlockRulesGroup:excludeRange


Best,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] "Timeout from remote TCP client" with dnsdist + PDNS Recursor

[Remi Gacogne via dnsdist]

Hi,

On 30/09/2022 17:56, Christian Joffre Calva Urrego via dnsdist wrote:
We currently use dnsdist 1.4 and have a PowerDNS Recursor configured as 
downstream server. Everything has a standard minimum configuration, with


..
client-tcp-timeout=60
..

configured on the PDNS Recursor.
The point is that in PDNS Recursor we can see the following entry log:

..
pdns_recursor: Timeout from remote TCP client 
..

We have been investigating and apparently the dnsdist does not close the 
TCP sessions with the configured downstream server (PDNS Recursor), 
having to close them the PDNS Recursor.


Is there a setting in dnsdist to set idle time of a TCP connection to a 
backend server?


I'm afraid there isn't.

What is the default lifetime of a TCP session in dnsdist, once it is 
waiting and not being used?


dnsdist only closes an idle outgoing TCP connection if it is already has 
'setMaxCachedTCPConnectionsPerDownstream' [1] idle connections to that 
backend in cache, or if the connection will not be reusable (because the 
proxy protocol is enabled, for example). Apart from that it tries to 
keep the connection alive for as long as possible, checking that the 
connection is still usable before trying to use it to forward a new query.
This is based on the reasoning that establishing a new TCP connection to 
a backend is not cheap and has an important cost in term of latency.


Are you experiencing any functional problem, apart from an entry in the 
recursor's logs? You did not share the version of the recursor you are 
using, but in recent version this warning is only written to the logs 
when 'log-common-errors' is set to 'yes'.


[1]: 
https://dnsdist.org/reference/tuning.html#setMaxCachedTCPConnectionsPerDownstream


--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] TCP / UDP backend queries

[Remi Gacogne via dnsdist]

Hi Thibaud,

On 30/09/2022 15:18, Thib D via dnsdist wrote:

Use case here is for authoritative DNS, not DoH.

As far as my understanding goes (and backed up by a tcpdump test),
a UDP query on the frontend will result on a UDP query to the backend 
server, and a TCP query on the frontend will result on a TCP query to 
the backend.


That's correct.


Is there a way to force dnsdist to make UDP queries to the backend or is 
this just not possible?


I'm afraid it's currently not possible to force dnsdist to make an 
outgoing query over UDP when the incoming query was received over TCP 
(the opposite is possible since 1.7.0 with the tcpOnly option on newServer).
I do not really want to implement that since the client would be 
confused by TC=1 replies received over TCP, DoT or DoH.
I am instead  considering implementing a mode where all outgoing queries 
are sent over UDP by default, only falling back to TCP if the answer is 
truncated, but this will not be before 1.9 so somewhere next year.



Best,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] dnsdist 1.7.2 released

[Remi Gacogne via dnsdist]

Hello!

We are very happy to release dnsdist 1.7.2 today, a maintenance release 
fixing a few bugs reported since 1.7.1:


- An unhandled exception could happen when an invalid protocol was used 
in an incoming DNS over HTTPS forwarded-for header and passed to the 
backend via the proxy protocol, leading to a use-after-free and a crash. 
Forwarded-for headers are not used by default and should only be used if 
the client can be trusted (#11667)
- An invalid proxy-protocol was sent to the backend, over TCP, if a 
query received via DNS over HTTPS resulted in a truncated UDP response 
from the backend (#11665)
- Some metrics lacked a proper description in our Prometheus endpoint 
(#11664)
- A side-effect of fixing the health-check timeout in 1.7.1 was leading 
to a CPU usage increase on devices that are mostly idle. We improved 
that situation, reducing the CPU usage even below what it was in 1.7.0 
(#11579, #11580)


We also added a couple Lua bindings to make it easier to look into the 
DNS payload from custom Lua rules and actions (#11666).


Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation.


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [3].


The release tarball [4] and its signature [5] are available on the 
downloads website, and packages for several distributions are available 
from our repository [6].


Finally, we would like to thank the PowerDNS community and all external 
contributors for their great work in this release!


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.7.2
[3]: https://github.com/PowerDNS/pdns/issues/new/choose
[4]:
https://downloads.powerdns.com/releases/dnsdist-1.7.2.tar.bz2
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.7.2.tar.bz2.sig
[6]: https://repo.powerdns.com

--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/



OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] dnsdist 1.7.1 released

[Remi Gacogne via dnsdist]

Hello!

We are very happy to release dnsdist 1.7.1 today, a maintenance release 
fixing a few bugs reported since 1.7.0:


- A use-after-free error could happen if a network error occurred in the 
middle of a XFR query, for a proxy-protocol-enabled backend, leading to 
a crash
- The TLS Server Name Indication was not properly set on outgoing DNS 
over HTTPS or DNS over TLS connections to a backend
- The health-check timeout was not properly set for outgoing DNS over 
HTTPS connections, leading to a very long timeout
- The outgoing protocol was not always properly set in our in-memory 
ring buffers
- Outgoing UDP timeouts were sometimes processed a bit too late when the 
health-check interval was set to more than one second

- Filtering qnames via eBPF was broken
- The dynamic block mechanism was not properly switching to eBPF 
filtering, when available, if the block action was not explicitly set

- The latency histogram was broken in our prometheus metrics
- Trying to create a 0-sized packet cache would lead to a crash

In addition to these fixes, our Docker images no longer have capability 
requirements. More information on that topic is available in our upgrade 
guide [1].


We also improved our compatibility with OpenSSL 3.0.0's API.

As usual there were also other smaller enhancements and fixes, please 
see the dnsdist website [2] for the more complete changelog [3] and the 
current documentation.


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [4].


The release tarball [5] and its signature [6] are available on the 
downloads website, and packages for several distributions are available 
from our repository [7].


Finally, we would like to thank the PowerDNS community and all external 
contributors for their great work in this release!


[1]: https://dnsdist.org/upgrade_guide.html#to-1-7-1
[2]: https://dnsdist.org
[3]: https://dnsdist.org/changelog.html#change-1.7.1
[4]: https://github.com/PowerDNS/pdns/issues/new/choose
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.7.1.tar.bz2
[6]:
https://downloads.powerdns.com/releases/dnsdist-1.7.1.tar.bz2.sig
[7]: https://repo.powerdns.com

--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Re: How to best handle DNS floods

[Remi Gacogne via dnsdist]

Hi,

On 06/04/2022 11:02, me aharen wrote:
Can you explain the "minimum of 10 answers during that time to reduce 
the risk of false-positive" part? Does it mean a minimum of 10 queries 
within that window, should be SERVFAIL?


It means that we need to have seen at least 10 answers, SERVFAIL or not, 
for that client during the 60s window. The idea is that we do not want 
to apply the ratio to a very small sample, because then there is a much 
bigger risk of false positive as the sample is not representative at all.


The dynamic rule can also use DNSAction.Pool instead 
of DNSAction.Truncate. How do I make use of the Pool? This way I could 
redirect them to a separate server.


Unless I'm mistaken I am afraid we do not support routing to a pool on a 
dynamic block match, because I do not see a way to pass the destination 
pool on a dynamic block rule. That sounds like a valid use-case, of 
course, so please feel free to open a feature request on GitHub and I'll 
try to implement that in the next version.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Best practice to handle massive DNS-JSON requests on DoH frontend

[Remi Gacogne via dnsdist]

Hi Pascal,

On 10/04/2022 19:15, Pascal K via dnsdist wrote:
We see massive DNS-JSON style requests on our DoH resolver which are 
correctly answered with HTTP 400 Bad Request [0] by dnsdist.


Here an example: GET /dns-query?name=asia1.ethermine.org

Every request comes in a new TCP connection forcing a new TLS handshake. 
So every connection a client sends ~1.6kB and our server responds with 
~6.1kB generating a lot of outbound traffic.


The IP sources are from all over the world and change after a few 
connection and seconds. Therefore, blocking the subnets of the IP origin 
does not work.


In my understanding the dnsdist rule engines does not help here because 
the HTTP listener already responses the 400 Bad Request.


Indeed, the HTTP 400 error is generated before the query reaches the DNS 
stack, so it does not make it to the rules engine.



We use dnsdist 1.4.0 on Ubuntu 20.04.

Any idea how to handle these DNS-JSON quereis to reduce our outbound 
network load?


I would need to see the actual traffic to be sure but our 400 errors are 
very small so I'm guessing most of the outbound traffic comes from the 
TLS certificate chain, meaning it's going to be complicated to do 
better, unless you can find something in the initial TCP packets or TLS 
handshake that can be used to identify that traffic.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Re: How to best handle DNS floods

[Remi Gacogne via dnsdist]

Hi,

On 03/04/2022 10:42, me aharen wrote:
Thanks for the input. Yes, we have legit customers participating in the 
PRSD floods.


Understood.

Setting the DynBlockRulesGroup:setRCodeRatio is interesting, can you 
share a sample config of this rule?


I cannot find any example in the documentation, which I really should 
fix, but we have a small example in our regression tests:


https://github.com/PowerDNS/pdns/blob/790f18878013eda17abb3fd5b0bc03cb87554c79/regression-tests.dnsdist/test_DynBlocks.py#L942

Basically to block for 120s any client that had say, more than 80% of 
Servfail answers over the last 60s, with a minimum of 10 answers during 
that time to reduce the risk of false positive, you would do:


local dbr = dynBlockRulesGroup()
dbr:setRCodeRatio(DNSRCode.SERVFAIL, 0.8, 60, "Exceeded servfail ratio", 
120, 10)


function maintenance()
  dbr:apply()
end

If you wanted to tell these clients to try again over TCP instead:

local dbr = dynBlockRulesGroup()
dbr:setRCodeRatio(DNSRCode.SERVFAIL, 0.8, 60, "Exceeded servfail ratio", 
120, 10, DNSAction.Truncate)


function maintenance()
  dbr:apply()
end

Regarding the professional service, whom should I contact to get pricing 
details?


The easiest option is likely to go to 
https://www.open-xchange.com/products/dnsdist/?hsLang=en and click 
"Contact OX", and someone from our team should get back to you quickly.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Proxy protocol question

[Remi Gacogne via dnsdist]

Hi Adrian,

On 02/04/2022 14:36, Adrian Kägi via dnsdist wrote:


Theese are my "newServer" statements:
newServer({address="pdns_auth_IPv4:5353", name="nsa-1_v4", pool="sec", 
useProxyProtocol=True})
newServer({address="[pdns_auth_IPv6]:5353", name="nsa-sec1_v6", 
pool="sec", useProxyProtocol=True})


Would you mind trying with useProxyProtocol=true instead of 
useProxyProtocol=True ? Lua is case sensitive and unless you define a 
variable named True whose value is true, it will not work:


$ lua
Lua 5.4.4  Copyright (C) 1994-2022 Lua.org, PUC-Rio
> a = true
> print(a)
true
> a = True
> print(a)
nil
>

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] How to best handle DNS floods

[Remi Gacogne via dnsdist]

Hi,

On 31/03/2022 10:59, me aharen via dnsdist wrote:
And added the action "addAction(RCodeRule(DNSRCode.SERVFAIL), 
DropAction())" - although I am uncertain if this works as I think it would.


This will not work as expected, as this rule is going to drop queries 
with a response code set to ServFail, not responses. If you want to drop 
responses instead, which I'm not sure is a very good idea, you need to do:


addResponseAction(RCodeRule(DNSRCode.SERVFAIL), DropResponseAction())

I do have another QPS rule, "addAction(MaxQPSIPRule(50), 
PoolAction("abuse"))", to redirect the flooders.


The only thing I can't do is apply any delay or drop action which would 
disrupt the user's legit queries.


So you have legitimate clients that take part in these DNS floods? Or 
are you concerned about false positive?


Using Dynamic Rule is interesting, but it blocks queries once the 
"exceedServFails" exceeds, blocks legit queries for /32 - which is 
disruptive.


You could use DynBlockRulesGroup:setRCodeRate() [1] to send a truncated 
answer to blocked clients instead of dropping their queries, so regular 
clients can retry over TCP. DynBlockRulesGroup:setRCodeRatio() [2] might 
even be better as it allows you to only block if the ratio of ServFail 
answers among all answers sent to a given client reaches a threshold.


If you are using a recent enough Linux kernel, it is even possible to do 
that with XDP for much better performance, see [3].


Note that it is also possible to use dnsdist to detect and block such 
PRSD attacks, with specific Lua scripts leveraging the ring buffers and 
dynamic blocks. Guidance and support for writing these scripts can be 
provided as part of our commercial solutions which include access to 
professional services.


[1]: 
https://dnsdist.org/reference/config.html#DynBlockRulesGroup:setRCodeRate
[2]: 
https://dnsdist.org/reference/config.html#DynBlockRulesGroup:setRCodeRatio

[3]: https://github.com/PowerDNS/pdns/pull/10498

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Is there some simple issue that needs to be done?

[Remi Gacogne via dnsdist]

Hi!

On 29/03/2022 14:52, Y7n05h wrote:

I'm interested in improving the performance of dnsdist with AF_XDP
in GSoC, I've spent a lot of time learning XDP and AF_XDP.
I hope there are some simple issues waiting to be done to help me get
familiar with the architecture of dnsdist.
It would be better if these issues could be related to XDP/AF_XDP.


I don't think we have any issue or feature request related to XDP or 
AF_XDP that are small enough for that.
There are a couple feature requests that are not very complicated from a 
code point of view and would make a good introduction to the dnsdist 
code at 
https://github.com/PowerDNS/pdns/issues?q=is%3Aopen+label%3Adnsdist+label%3Aeasy


If you are looking for bigger things to tackle, perhaps there might be 
of interest to you:


- https://github.com/PowerDNS/pdns/issues/11301
- https://github.com/PowerDNS/pdns/issues/10793
- https://github.com/PowerDNS/pdns/issues/10769
- https://github.com/PowerDNS/pdns/issues/9690
- https://github.com/PowerDNS/pdns/issues/11261

Please drop a comment either here or directly on the related GitHub 
issue if you start working on something to avoid duplicated work.



I tried to contact Remi Gacogne a long time ago, but he may be busy.
He did not reply to my email.


I'm usually not working on PowerDNS-related topics during week-ends, and 
I might indeed need more than a few days to reply to non-urgent e-mails 
during some busy periods.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist[29321]: Marking downstream IP:53 as 'down'

[Remi Gacogne via dnsdist]

Hi,

> We have configured dnsdist instance to handle around 500k QPS, but we
> are seeing downstream down frequently once QPS reached above 25k. below
> are the logs which we found to relative issue.
>
> dnsdist[29321]: Marking downstream server1 IP:53 as 'down'
>
> dnsdist[29321]: Marking downstream server2 IP:53 as 'down'

You might be able to get more information about why the health-checks 
are failing by adding setVerboseHealthChecks(true) to your configuration.


It usually happens because the backend is overwhelmed and needs to be 
tuned to handle the load, but it might also be caused by a network 
issue, like a link reaching its maximum capacity, or by dnsdist itself 
being overwhelmed and needing tuning (like increasing the number of 
newServer() directives, see [1]).


[1]: 
https://dnsdist.org/advanced/tuning.html#udp-and-incoming-dns-over-https


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Re: How to apply dynamic rules with pools?

[Remi Gacogne via dnsdist]

On 23/02/2022 17:20, Willis, Michael wrote:

I changed the to rule to:
dbr:setQTypeRate(DNSQType.ANY, 1, 10, "Exceeded ANY rate", 600)
After testing It looks like the entire 10 seconds needed to elapse 
before the rule is evaluated.
I was not expecting this logic, and that was tripping me up. I was 
thinking that the rules were not applying at all.


For low rates it does indeed often take the whole interval to trigger, 
but it might trigger more quickly than that depending on the traffic. 
For example if one client sends 101 ANY queries per second during one 
second, it should be enough to trigger your rule almost immediately.


I can see how that's surprising, if you think of any way we can improve 
the documentation on that point please let us know! Pull requests are of 
course very welcome, but even suggesting a few lines to add to the 
documentation on this mailing-list already helps a lot :)


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] How to apply dynamic rules with pools?

[Remi Gacogne via dnsdist]

Hi Mike,

On 23/02/2022 16:49, Willis, Michael via dnsdist wrote:
I have intentionally set the trigger for "ANY" to 1 ever 100 seconds, so 
it will trigger and stay triggered.

This is so I can verify the correct rule is applying.



dbr:setQTypeRate(DNSQType.ANY, 1, 100, "Exceeded ANY rate", 600)


This rule is saying "block, for 600 seconds,  clients that have been 
sending more than one ANY query per second over the last 100 seconds", 
so one query is not going to be enough to trigger the block.


You could try this one instead:

dbr:setQTypeRate(DNSQType.ANY, 0, 1, "Exceeded ANY rate", 600)

This will block any client that has been sending more than 0 ANY query 
per second over the last second. In my test this results in getting 
blocked right away after sending your first ANY query. I'm not sure I 
would recommend such a drastic rule, but that's a different matter :)


Hope that helps,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist and powerdns on same machine

[Remi Gacogne via dnsdist]

Hi Stephan,

On 04/02/2022 10:47, De Webmakers (Stephan) via dnsdist wrote:


I’ve been struggling with this for far to long now…

Is it possible to run dnsdist and pdns on the same server and accept dns 
request from everyone (just as it would be without dnsdist).


The problem is that I just can’t seem to find a good dnsdist.conf 
example to work with and that’s working.


In my mind it should be as simple as changing the port for pdns to let’s 
say 5300 and then adding a 127.0.0.1:5300 as server to dnsdist..


However the nameserver becomes instantly unreachable after this.

Can anyone point me in the right direction?


Well it is, usually, as simple as that, so if that's not working for you 
I'm afraid you will have to tell us more about your exact setup. Sharing 
the configurations of both dnsdist and pdns would be a good start.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist and PROXYv2 testing - accepting TCP connections when an upstream server is available

[Remi Gacogne via dnsdist]

Hi Oto,

On 31/01/2022 16:50, Oto Šťáva via dnsdist wrote:
firstly, I want to thank everyone involved for making dnsdist available, 
it has helped me greatly these past few weeks with implementing and 
testing support for the PROXYv2 protocol in Knot Resolver [1] here at 
CZ.NIC.


That's very good news, thanks!

I have been looking through the dnsdist docs and what I am looking for 
is probably currently not available, but I would like to ask first 
before deciding on what to do next: Is there a way to make dnsdist 
refuse all TCP connections while there are no upstream servers 
available?


I'm afraid there is not. The reason why is that dnsdist supports 
multiple pools of servers, and queries can be routed to a different pool 
based on their provenance and/or content, so we cannot really know 
beforehand whether we will be able to handle a TCP connection or not. We 
could even be able to serve some queries but not others sent on the same 
TCP connection, depending on the configured rules.


If not, do you have any suggestions on how to properly detect 
dnsdist's readiness?


If you already track the state of the backends and can wait until they 
are actually ready before starting the test, you might be able to get 
away with simply disabling health-checking in dnsdist by setting the 
server status to 'Up' [1].


Otherwise the more reliable way to do that would be to use the REST API 
[2] to get the status of the pool you are interested in, but that would 
require writing a bit of dnsdist-specific code which is not great in 
your case.
Would sending an actual UDP query be a possibility? You could then 
detect the availability by checking that you get either a response 
(default) or a non-servfail answer (if you set 
setServFailWhenNoServer(true) [3]).




[1]: https://dnsdist.org/reference/config.html?highlight=setup#Server:setUp
[2]: 
https://dnsdist.org/guides/webserver.html?highlight=api#get--api-v1-servers-localhost-pool?name=pool-name

[3]: https://dnsdist.org/guides/serverselection.html#setServFailWhenNoServer

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] forwarding domains to special servers

[Remi Gacogne via dnsdist]

Hi Thomas,

On 21/01/2022 13:55, Thomas Mieslinger via dnsdist wrote:

I'm completely new to dnsdist. I'd like to use it for DNS split horizon
setup.

Goal: send queries which end with 'internal.domain' to Pool "int".

According to the documentation there are two ways to do so:

-- setup default pool
newServer({address="2001:8d8:fe:53:72ec::1"})
-- internal nameserver
newServer({address="fd7c:b55:e086:41:500c::1", pool="int"})

Possibility 1:

addAction({'internal.domain'}, PoolAction("int"))

Possibility 2:

InternalDomains = newSuffixMatchNode();
InternalDomains:add(newDNSName("internal.domain"));
addAction(SuffixMatchNodeRule(InternalDomains), PoolAction("int"));

Are there rules of thumb which possibility to prefer?


Both possibility do exactly the same thing under the hood.
Personally I would only use the second one if you want to be able to 
alter the list at runtime, via the console.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] XDP/eBPF blocking (was dnsdist 1.7.0 released)

[Remi Gacogne via dnsdist]

Hi Klaus,

On 17/01/2022 21:05, Klaus Darilion wrote:
Pierre Grié from Nameshield contributed an XDP program to reply to 
blocked UDP queries with a truncated response directly from the

kernel, in a similar way to what we were already doing using eBPF
socket filters. This version adds support for eBPF pinned maps,
allowing dnsdist to populate the maps using our dynamic blocking
mechanism, and letting the external XDP program do the actual
blocking or response.


How does this work in detail? If example.com is on these lists
(filtering or truncate response), will it block also www.example.com
(and other subdomains) or only exactly the name on the list?


I'm afraid the current XDP program would only block the exact name on 
the list. Now that the actual program can live outside of dnsdist it 
would be easier to write a new XDP program doing suffix matching, but 
no-one has done so yet. The main issue was that we wanted to keep our 
eBPF code working on older kernels where the number of eBPF instructions 
is very limited, but it would be very fine for an external XDP program 
to target newer kernels only. I would be happy to merge such a program 
in our "contrib" directory :-)


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] dnsdist 1.7.0 released

[Remi Gacogne via dnsdist]

Hi everyone!

We are proud to announce the release of dnsdist 1.7.0. This release 
contains several new exciting features since 1.6.1, as well as 
improvements and bug fixes. It contains one single change from the first 
release candidate, a fix for DynBlockRatioRule::warningRatioExceeded 
provided by Doug Freed.


In our view, the most exciting new feature of 1.7.0 is the support of 
outgoing DNS over TLS and DNS over HTTPS, as well as the ability to do 
"cross-protocol" queries, meaning a query received over a given protocol 
(UDP, TCP, DoT, DoH, ...) can be forwarded over a different one. Now 
that dnsdist is capable of contacting its backend over an encrypted 
channel, full end-to-end encryption is possible, offering improved 
confidentiality and integrity.


Among the new features is the ability to add a custom EDNS option to a 
query before forwarding it to a backend, via SetEDNSOptionAction. 
phonedph1 also contributed a new rule making it possible to route a 
query based on the number of outstanding queries in a pool, 
PoolOutstandingRule.


Pierre Grié from Nameshield contributed an XDP program to reply to 
blocked UDP queries with a truncated response directly from the kernel, 
in a similar way to what we were already doing using eBPF socket 
filters. This version adds support for eBPF pinned maps, allowing 
dnsdist to populate the maps using our dynamic blocking mechanism, and 
letting the external XDP program do the actual blocking or response.


The packet cache has been improved so that one can now configure which 
EDNS options should be ignored, raising the cache hit ratio behind 
customer-premises equipment. The incoming and outgoing protocols have 
been added to the output of the grepq command for a better understanding 
of the recently processed traffic.


Dimitrios Mavrommatis improved the handling of AXFR and IXFR queries, 
making it possible to reuse a TCP connection used for a zone transfer 
much more efficiently.


We added support for generating the still experimental SVCB and HTTPS 
records directly from dnsdist, offering potential benefits to both 
performance and privacy.


Our LMDB code has gained the ability to do range-based lookups, and is 
now more performant even for simple lookups.


Extending the per-thread custom load-balancing policies introduced in 
1.6.0, it is now possible to write blazing-fast, lock-less per-thread 
custom actions using the Lua foreign function interface.


Holger Hoffstätte also improved the reporting of an unavailable backend, 
making sure the existing metrics are no longer reported to prevent any 
confusion.


This release also reduces the memory footprint of dnsdist in several 
places, which makes it easier to use in resource-constrained environments.


Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation.


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [3].


The release tarball [4] and its signature [5] are available on the 
downloads website, and packages for several distributions are available 
from our repository [6].


With this release, the 1.4.x releases become EOL and the 1.5.x and 1.6.x 
releases go into critical security fixes only mode.


Finally, we would like to thank the PowerDNS community and all external 
contributors for their great work in this release!


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.7.0
[3]: https://github.com/PowerDNS/pdns/issues/new/choose
[4]:
https://downloads.powerdns.com/releases/dnsdist-1.7.0.tar.bz2
[5]:
https://downloads.powerdns.com/releases/dnsdist-1.7.0.tar.bz2.sig
[6]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] frontend responses for resolver timeouts

[Remi Gacogne via dnsdist]

Hi David,

On 07/01/2022 18:03, David Bader via dnsdist wrote:
Ok, so in case of a timeout, dnsdist does not send anything to the 
client and the client will also timeout (and retry).


Does that mean, it would make sense to increase the dnsdist 
configuration to use the same timeout as the client uses?

If we are in the rare position to know the client's timeout.
(dnsdist is not under heavy load)


It would make sense if you know the client's timeout duration and do 
understand why the backend takes so long to answer, yes.


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] frontend responses for resolver timeouts

[Remi Gacogne via dnsdist]

Hi David,

On 07/01/2022 12:06, David Bader via dnsdist wrote:

Hello,

is my understanding correct, that dnsdist sends the client a
ServFail answer after 2 seconds when the backend resolver does not 
respond within the timeout (2 seconds by default):

https://www.dnsdist.org/reference/tuning.html#setUDPTimeout

and the frontend-servfail counter is increased by one?
https://www.dnsdist.org/statistics.html#frontend-servfail


I'm afraid dnsdist doesn't a ServFail when the backend resolver does not 
respond within the UDP timeout, no, it just records that the query was 
not answered in time and increases the 'downstream-timeouts' counter. 
Not sending a ServFail should lead the client to retry, giving the 
backend a second chance now that the answer might be (partially?) in cache.


dnsdist will answer with a ServFail if there is no backend available, 
for example because the backend does not properly respond to 
health-check queries, provided that setServFailWhenNoServer is set [1]. 
In that case both the 'no-policy' and 'frontend-servfail' counters will 
be increased.


[1]: https://dnsdist.org/guides/serverselection.html#setServFailWhenNoServer

Hope that helps,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] no ipv6 connectivity

[Remi Gacogne via dnsdist]

Hi Larry,

On 06/01/2022 18:16, Larry Wapnitsky via dnsdist wrote:

I've set up dnsdist in my lab to forward to my dns servers, running powerdns

If I do nslookup directly to the ipv6 addresses on the name servers, I 
can resolve.


If I try to resolve via dnsdist, I get no connection.

Here is my config, modified to avoid public addresses:

setLocal('10.150.33.2')
addLocal('2001:xxx::33::2')


Did you change the default ACL? It allows 10.0.0.0/8 but only a few IPv6 
prefixes:

- ::1/128
- fc00::/7
- fe80::/10

https://dnsdist.org/advanced/acl.html

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] First release release of dnsdist 1.7.0

[Remi Gacogne via dnsdist]

Hi everyone!

We are happy to announce the first release candidate of what will become 
dnsdist 1.7.0, with only one fix and one improvement since the second beta.


We fixed a crash introduced in 1.7.0-alpha1 that could occur when a DoH 
query was forwarded to a backend over TCP, DoT or DoH and the response 
was dropped by a rule.


We also improved the health-checks queries done over DoT so that we 
could use any cached TLS ticket when connecting to the server, but also 
save new tickets so that they can be used for later connections. That 
reduces the CPU load and improves response time on devices dealing with 
a low number of queries per second.


Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation.


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [3].


The release tarball [4] and its signature [5] are available on the 
downloads website, and packages for several distributions are available 
from our repository [6].


With the future 1.7.0 final release, the 1.4.x releases will be EOL and 
the 1.5.x and 1.6.x releases will go into critical security fixes only mode.


Finally, we would like to thank the PowerDNS community and all external 
contributors for their great work in this release!


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.7.0-rc1
[3]: https://github.com/PowerDNS/pdns/issues/new/choose
[4]:
https://downloads.powerdns.com/releases/dnsdist-1.7.0-rc1.tar.bz2
[5]:
hhttps://downloads.powerdns.com/releases/dnsdist-1.7.0-rc1.tar.bz2.sig
[6]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Large domain list blocking via DNS

[Remi Gacogne via dnsdist]

Hi Jahanzeb,

On 13/12/2021 08:00, Jahanzeb Arshad via dnsdist wrote:
We want to implement blocking of large number (3M+) of undesirable 
domains (adult/malware) via DNS. We have tested using PowerDNS recursor 
and it is working in test environment. For blocking we have use LUA dns 
script with domains in sqlite3 DB. Can we do same on DNSDIST as we are 
using it as load balancer on front of all our resolver DNS. Does DNSDIST 
support LUA with sqlite3 or other database.


The Lua code would be quite different in dnsdist but you can do 
something similar using a LuaAction [1] and DNSAction.Spoof [2], like we 
do in our regression tests [3].


I would suggest considering different options than a sqlite database if 
you are dealing with a huge number of queries, though. dnsdist provides 
native support for looking into a CDB or LMBD database, for example [4].


[1]: https://dnsdist.org/advanced/luaaction.html
[2]: https://dnsdist.org/reference/constants.html#dnsaction
[3]: 
https://github.com/PowerDNS/pdns/blob/7285a16693f67287c0d8151afbcefb7190b63cf3/regression-tests.dnsdist/test_Spoofing.py#L406

[4]: https://dnsdist.org/reference/kvs.html

Hope that helps,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Dynamic Rule for abusive SERVFAIL queries from bots

[Remi Gacogne via dnsdist]

Hi,

On 11/12/2021 08:44, me aharen via dnsdist wrote:
I am running  dnsdist 1.6.1 and I am unable to figure out the safest 
method of handling large amounts of SERVFAIL queries to random domains.


Right now I manually check SERVFAIL responses via 'topResponses(50, 
dnsdist.SERVFAIL)', and pick a repeating domain from the list and then 
apply a LogAction to identify the offending IPs.


After some log collection, I run through sort/unique and pick the IPs 
that are cache poisoning with SERVFAILs and simply apply rule 
'addAction(, PoolAction("abuse"))'.


It a very manual cumbersome approcah. The best method I found was to use 
dynBlockRulesGroup() as follows:


local dbr = dynBlockRulesGroup()

dbr:setRCodeRate(DNSRCode.SERVFAIL, 20, 10, "Exceeded ServFail
rate", 60)


function maintenance()

dbr:apply()

end

This works but this is a bit disruptive. I would like is to send the 
offending IP's to an abuse pool I have already set up.


Any Idea how can use the dynBlock function to send the IP's to abuse 
Pool? or is there a better way to handle this?


I'm afraid dnsdist currently doesn't support routing to a pool as a 
target of a dynamic block, that might be an interesting feature request. 
Or perhaps adding the offending IP to a NetmaskGroup so that it can be 
used from existing rules. Or both :)


One way to reduce the risk of false positives might be to rely on the 
ratio of ServFail answers over all the answers returned to a given 
client, instead of just using the raw numbers of ServFail answers, which 
can be done with DynBlockRulesGroup:setRCodeRatio [1]. That way you can 
block a client if it received more than N servfails AND more than Y % 
percent of all responses for that client were servfails.


[1]: 
https://dnsdist.org/reference/config.html?highlight=setrcode#DynBlockRulesGroup:setRCodeRatio


Hope that helps,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Tune DNSDIST for proper traffic diversion and caching for lower latency

[Remi Gacogne via dnsdist]

Hi Chandra,

On 10/12/2021 14:27, Chandra via dnsdist wrote:
For #1: I didn't find a proper server policy to fit my needs but, it 
doesn't seem to be a completely new thing to have. Currently the 
weighted random policy does work to some extent. But there are some 
queries which goto the fallback servers, for eg out of 30k queries at 
least 50 of them goto the fallback servers, I do not want this. Is there 
a way to achieve what I described in #1?


I would suggest using two different pools of servers and the 
PoolAvailableRule rule [1]. That way queries will be always routed to 
the first pool if it has at least one server available, and to the 
second one otherwise.
If you don't set any cache on the second pool the answers coming from 
these servers won't be cached. The only part of your requirements that I 
don't see how to address is the "no stale cache" part, since we will 
only check for stale entries in the cache associated to the pool that 
was selected, and therefore will not check the cache for the first pool 
when its servers are down.


[1]: https://dnsdist.org/rules-actions.html#PoolAvailableRule

For #2:  This is the most concerning issue for me at the moment, the 
average latency is about 80 ms (10k packet average), where as my primary 
server latency is much lower (~50ms) and most confusing part is the 
packet cache stats:


Entries: 86/1
Hits: 4894
Misses: 21543
Deferred inserts: 0
Deferred lookups: 0
Lookup Collisions: 0
Insert Collisions: 0
TTL Too Shorts: 0

I was under the impression that if there's a cache miss then the 
downstream response will be cached. Testing my setup for a couple of 
days, I have never seen my cache crossing 100.  Why is the response not 
being cached, where there's a miss. Here are the current  extended stats:


I'm afraid it's hard to know without knowing exactly which answers are 
not cached. There are some answers that dnsdist will refuse to cache:
- when the minimum TTL is lower than minTTL, but since minTTL is set to 
0 in your configuration that should not be it ;
- when there is no records at all in the answer and the rcode is 
different from ServFail or Refused, since we don't know for how long to 
keep that entry.


I would suggest having a look at 'grepq("")' to try to understand which 
answers are not cached at the next step.


from what I see, there are a lot of udp errors. How to fix this? Also to 
add: all my traffic is udp based, I am not accepting TCP traffic yet.


Please note that some values are system-wide counters and might not be 
related to dnsdist at all, like udp-* and udp6-*.


"noport" errors cannot be fixed on the server side, they usually mean 
that the remote end closed its socket before receiving the response. 
"in-errors" might be caused by dnsdist not accepting UDP queries fast 
enough, which can be fixed by tuning your configuration [2] [3].


[2]: 
https://dnsdist.org/reference/tuning.html?highlight=recv#setUDPSocketBufferSize
[3]: 
https://dnsdist.org/advanced/tuning.html#udp-and-incoming-dns-over-https


Best,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] Second beta release of dnsdist 1.7.0

[Remi Gacogne via dnsdist]

Hi everyone!

We are happy to announce the second beta release of dnsdist 1.7.0, with 
few fixes since the first beta, the most important one being a memory 
leak when reusing TLS sessions for outgoing DNS over TLS and DNS over 
HTTPS connections. During that work we stumbled upon a memory leak in 
some setups using GnuTLS which will have to be fixed in the library 
itself. After reporting it upstream we added a warning in dnsdist which 
will be removed when a fixed version of GnuTLS has been released.


We also fixed an error in the way we check for integer overflows in 
configuration values, which could have refused valid configurations.


Finally we added a function to see the current configuration of the 
internal web server.


Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation.


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [3].


The release tarball [4] and its signature [5] are available on the 
downloads website, and packages for several distributions are available 
from our repository [6].


With the future 1.7.0 final release, the 1.4.x releases will be EOL and 
the 1.5.x and 1.6.x releases will go into critical security fixes only mode.


Finally, we would like to thank the PowerDNS community and all external 
contributors for their great work in this release!


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.7.0-beta2
[3]: https://github.com/PowerDNS/pdns/issues/new/choose
[4]:
https://downloads.powerdns.com/releases/dnsdist-1.7.0-beta2.tar.bz2
[5]:
hhttps://downloads.powerdns.com/releases/dnsdist-1.7.0-beta2.tar.bz2.sig
[6]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] First beta release of dnsdist 1.7.0

[Remi Gacogne via dnsdist]

Hi everyone!

We are happy to announce the first beta release of dnsdist 1.7.0!

We introduced a fair number of improvements and new features since the 
second alpha, and we will now iron out the documentation and fix any 
bugs before hopefully releasing the first release candidate very soon.


The main new feature is the ability to use the same outgoing TCP or DNS 
over TLS connection for queries coming from different clients, leading 
to a huge decrease of the number of outgoing connections needed when the 
backend supports out-of-order processing.


We also added the exact transport type to dnstap and protocol buffer 
messages, making it possible to differentiate between plaintext queries 
and DNS over HTTPS or DNS over TLS ones.


Recently Pierre Grié from Nameshield contributed an XDP program to reply 
to blocked UDP queries with a truncated response directly from the 
kernel, in a similar way to what we were already doing using eBPF socket 
filters. This beta finally adds support for eBPF pinned maps, allowing 
dnsdist to populate the maps using our dynamic blocking mechanism, and 
letting the external XDP program do the actual blocking or response.


Stéphane Bortzmeyer helped us pinpoint a few issues in the encryption 
between dnsdist and its backends, notably in the way the outgoing 
connections are cached while waiting to be reused. That could have led 
to a waste of memory piling up over time.


We also fixed an issue where the threads handling incoming DoH queries 
could have stopped processing responses when they were completely 
overloaded by TLS handshakes, leading to a degradation of performance.


The last issue was that a backend was not properly marked as 
non-available when a certain exception was raised during a health-check 
attempt.


Finally Rosen Penev contributed a lot of clean up changes to make sure 
that we make the best of what C++17 can offer.


Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation.


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub [3].


The release tarball [4] and its signature [5] are available on the 
downloads website, and packages for several distributions are available 
from our repository [6].


With the future 1.7.0 final release, the 1.4.x releases will be EOL and 
the 1.5.x and 1.6.x releases will go into critical security fixes only mode.


Finally, we would like to thank the PowerDNS community and all external 
contributors for their great work in this release!


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.7.0-beta1
[3]: https://github.com/PowerDNS/pdns/issues/new/choose
[4]:
https://downloads.powerdns.com/releases/dnsdist-1.7.0-beta1.tar.bz2
[5]:
hhttps://downloads.powerdns.com/releases/dnsdist-1.7.0-beta1.tar.bz2.sig
[6]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist using loopback address as source address for queries

[Remi Gacogne via dnsdist]

Hi Adam,

On 11/12/21 04:05, Adam Bishop via dnsdist wrote:> 'print *dss' didn't 
work, but dss looked like it contained a smart

pointer, so I tried 'print *dss._M_ptr - the output of that is at the
end of this message. The field seems intact though.


That's very useful, thank you. Indeed I don't see any unexpected content 
in there, the source interface and source address are unset, the 
destination address is correctly set, nothing that would explain why we 
suddenly get a socket connected with a different source.



It also happens on a health-check failure when
'reconnectOnFailure=true' is set on a backend, but that's not the
case in your configuration. Actually, I wonder if that would make a
difference, do you think you could try setting that option on one
of the IPv6 backends?


It's tricky to tell because the issue only happens infrequently, but
my impression is after letting the systems run for several days is
that this has had an effect. I can say definitively though that the
back end going away is the proximal cause - I bounced one of the
backends up and down and I was able to trigger failures that
coincided with the backend going down. There must be an additional
timing or duration factor though, as I've not yet been able to
trigger the issue on demand.


Depending on how quickly that happens when you bound the backend up and 
down, do you think you might be able to strace the dnsdist process at 
the same time?


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist to limit requests per domain

[Remi Gacogne via dnsdist]

Hi Stephan,

On 11/8/21 13:03, De Webmakers (Stephan) via dnsdist wrote:

We recently experienced a DDoS on our nameservers.

We are now looking to (help) prevent this in the future and since we are 
using powerDNS we came across dnsdist.


We analyzed the DDoS requests and the requests came from different 
(probably spoofed) IP’s.


For example x.y.z.1 and then x.y.x.2 etc.

The requested domain was the same every time with a different subdomain.

For example a.example.com and then b.example.com.

Would it be possible for dnsdist to limit requests per domain instead of 
per IP?


So if there are more then 10 requests for *.example.com in a second (or 
something) the requests for that entire domain (example.com) are dropped 
for 60 seconds (or more).


The building blocks to detect and mitigate PRSD attacks are there, from 
the information in the ring buffers about recent queries and responses 
to dynamic block rules, but that requires writing quite a bit of Lua to 
tailor the behaviour to your needs. Our professional services have done 
that work for several customers already.


It is also possible to do more simple rate-limiting per domain using a 
SuffixMatchNodeRule [1] (which is much more efficient than a regular 
expression) combined with a MaxQPSRule [2], for example.


[1]: https://dnsdist.org/rules-actions.html#SuffixMatchNodeRule
[2]: https://dnsdist.org/rules-actions.html#MaxQPSRule

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist using loopback address as source address for queries

[Remi Gacogne via dnsdist]

On 10/29/21 15:32, Adam Bishop via dnsdist wrote:

On 29 Oct 2021, at 13:38, Remi Gacogne via dnsdist 
 wrote:

Would you mind checking that you still have IPv6 addresses on that interface? I 
see you still have some on the incoming interface, though, since we receive a 
query over IPv6 on file descriptor 18 in your strace output.


Yeah, it's a single interface system so the ingress/egress addresses are the 
same.


Any events in the system logs that looks like the IP addresses might have 
changed at some point?


The only thing in dmesg is an SElinux policy update triggered by dnf, but no 
AVC entries that would indicate that being the cause. NetworkManager hasn't 
written anything to its own log since boot, which is normal.


Anything in the dnsdist logs looking like a reconnect (error while writing to 
the backend, ..)? We should not reconnect unless send() failed with EINVAL or 
ENODEV, which is not supposed to happen in your case since you don't set the 
source interface.


There's some normal terminations when the backend is reloaded for config 
changes, e.g.

   dnsdist[1351]: Timeout while waiting for the health check response from 
backend [2001:630:1:170::67]:53
   dnsdist[1351]: Marking downstream [2001:630:1:170::67]:53 as 'down'
   dnsdist[1351]: Timeout while waiting for the health check response from 
backend 193.63.72.67:53
   dnsdist[1351]: Marking downstream 193.63.72.67:53 as 'down'
   dnsdist[1351]: Marking downstream [2001:630:1:170::67]:53 as 'up'
   dnsdist[1351]: Marking downstream 193.63.72.67:53 as 'up'

But that's it - the only other messages are "Packet from:"


Alright, thanks for checking! I have to say I'm running a bit out of 
ideas. We only bind to a specific source address if one is supplied in 
the configuration, and that's not the case here. The field holding that 
source address is not updated at runtime, and I would be surprised to 
see a memory corruption touching only that field. You might actually be 
able to check that via gdb, by attaching to the process, selecting the 
right thread with 'thread 7' (it needs to be one of these in 
responderThread() for the right backend, which is 7 or 9 in your current 
process based on the gdb output) then 'up' three times to reach the 
responderThread frame, and 'print *dss' to see the content of the structure.
Even if that field is corrupted, we would need to reconnect which 
normally does happen only the errors I mentioned earlier, and we don't 
have these in your logs. It also happens on a health-check failure when 
'reconnectOnFailure=true' is set on a backend, but that's not the case 
in your configuration. Actually, I wonder if that would make a 
difference, do you think you could try setting that option on one of the 
IPv6 backends?


Best regards,

Remi


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist using loopback address as source address for queries

[Remi Gacogne via dnsdist]

On 10/29/21 13:04, Adam Bishop via dnsdist wrote:

On 29 Oct 2021, at 11:33, Remi Gacogne via dnsdist 
 wrote:

would you mind getting the output of "lsof -n -p " while the 
issue is still happening? A full backtrace with gdb might also be good to have


Sure that's fine, output follows. I've uploaded the gdb output as it's quite 
long. We've got three other instances to take the load, so I'm happy to keep it 
running as is for the moment.


Thanks! So we see from the strace output that dnsdist is calling send() 
(which becomes the sendto() syscall) to send the queries to the backend, 
using the connected UDP sockets:


[pid  1384] sendto(11, "U\251\0 
\0\1\0\0\0\0\0\1\4\4lbdn\4virt\2ja\3net\0\0\6\0\1\0\0)\20\0\0\0\0\0\0\30\0\10\0\24\0\2\200\0 
\1\0060\0
[...]
[pid  1382] sendto(11, "V\251\0 
\0\1\0\0\0\0\0\1\4\4lbdn\4virt\2ja\3net\0\0\6\0\1\0\0)\20\0\0\0\0\0\0\f\0\10\0\10\0\1 
\0\302S\7\305", 62, 0, NULL, 0) = 62


But we then see in the lsof output that these IPv6 sockets (the 11 is 
used above, but the others are the same) are connected using ::1 as 
source, while the IPv4 ones are not:



dnsdist 1351 dnsdist   11u IPv6  28637  0t0  UDP 
[::1]:35903->[2001:630:1:160::195]:domain
dnsdist 1351 dnsdist   13u IPv4  28640  0t0  UDP 
194.83.7.197:45941->194.83.7.195:domain
dnsdist 1351 dnsdist   15u IPv6  28650  0t0  UDP 
[::1]:51827->[2001:630:1:170::67]:domain
dnsdist 1351 dnsdist   17u IPv4  28653  0t0  UDP 
194.83.7.197:57454->193.63.72.67:domain


That explains what you are seeing, but I don't understand how we end up 
with these..
Would you mind checking that you still have IPv6 addresses on that 
interface? I see you still have some on the incoming interface, though, 
since we receive a query over IPv6 on file descriptor 18 in your strace 
output. Any events in the system logs that looks like the IP addresses 
might have changed at some point? Anything in the dnsdist logs looking 
like a reconnect (error while writing to the backend, ..)? We should not 
reconnect unless send() failed with EINVAL or ENODEV, which is not 
supposed to happen in your case since you don't set the source interface.


Best regards,

Remi


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] dnsdist using loopback address as source address for queries

[Remi Gacogne via dnsdist]

Hi Adam,

On 10/29/21 12:10, Adam Bishop via dnsdist wrote:

On 13 Sep 2021, at 13:47, Adam Bishop  wrote:

On 13 Sep 2021, at 13:31, Remi Gacogne via dnsdist 
 wrote:

That's very weird, I don't have any clue to what might be happening. Would you 
mind sharing the whole configuration? In particular, do you set the source 
interface? I would also be very interested in seeing a strace of the process 
while the issue is happening.


Sure, I can attach strace the next time I catch it in the act. It's occurring 
infrequently, but reliably so might take me a few days. Source interface is not 
explicitly set, as the host is single homed.


Took a while to recur, but I attached trace. Nothing immediately jumps out at 
me, there's no errors returned by the socket.

Command used was: strace -v -s 1024 -Ff -p 1351 2>&1 | grep -v epoll_wait | 
grep -v nanosleep

I've left the instance in the broken state, so if there's additional testing 
that would be useful, please let me know.


I'll look at the trace more closely in a bit but would you mind getting 
the output of "lsof -n -p " while the issue is still 
happening? A full backtrace with gdb might also be good to have, you 
would first need to attach to the running process with 'gdb -p dnsdist' then issue the 'thread apply all bt full' command. You can then 
close gdb and let dnsdit running (or not).


Thanks!


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] Modify EDNS NSID response back to client

[Remi Gacogne via dnsdist]

Hi Rob,

On 10/21/21 18:13, Robert Schwartz via dnsdist wrote:
I'm trying to find a way to add a suffix to an existing NSID string 
response coming back from a backend server. My use-case is to be able to 
know, not only which backend server responded to the query (that's in 
the backend response), but to also know which DNSDist box handled the 
proxy request. The DNSDist boxes are behind an anycast IP (aws global 
accelerator) so it's not always clear which DNSDist box handled a given 
request.


Is this possible and how would I go by implementing this?


I'm afraid dnsdist doesn't offer any way to do that easily, short of 
parsing and rewriting the whole DNS payload in Lua by yourself.
As a rule we tend to avoid adding code to alter responses, as we believe 
dnsdist is not really well-suited to that job.
In your case I see how it might be difficult to do that in the backend 
itself, of course :-/


Best,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

[dnsdist] Second alpha release of dnsdist 1.7.0

[Remi Gacogne via dnsdist]

Hi everyone,

We are happy to announce the second alpha release of dnsdist 1.7.0!

We spent quite some time since alpha1 reproducing an issue reported by 
Stephane Bortzmeyer in our new outgoing DNS over TLS feature. The issue 
turned out to be triggered by the use of the GnuTLS provider, and to be 
only present with some versions of that library. We are still working 
with the GnuTLS project to get this issue resolved, but in the meantime 
we implemented a work-around in dnsdist itself. In addition to that 
work-around, this release contains a few new features, improvements and 
bug fixes.


Among the new features is the ability to add a custom EDNS option to a 
query before forwarding it to a backend, via SetEDNSOptionAction. 
phonedph1 also contributed a new rule making it possible to route a 
query based on the number of outstanding queries in a pool, 
PoolOutstandingRule.


The packet cache has been improved so that one can now configure which 
EDNS options should be ignored, raising the cache hit ratio behind 
customer-premises equipment. The incoming and outgoing protocols have 
been added to the output of the grepq command for a better understanding 
of the recently processed traffic. We also reduced the memory 
consumption of dnsdist in constrained environments a bit further.


Denis Machard reported that queries received over UDP and forwarded via 
a TCP, DoH or DoT were not properly cached. We also noticed that the 
includeDirectory configuration directive might not properly function if 
an exception was raised during the processing. Both issues are now fixed.


Please see the dnsdist website [1] for the more complete changelog [2] 
and the current documentation.


Please send us all feedback and issues you might have via the mailing 
list, or in case of a bug, via GitHub.


Release tarballs are available on the downloads website, and packages 
for several distributions are available from our repository.


With the future 1.7.0 final release, the 1.4.x releases will be EOL and 
the 1.5.x and 1.6.x releases will go into critical security fixes only mode.


Finally, we would like to thank the PowerDNS community and all external 
contributors for their great work in this release!


[1]: https://dnsdist.org
[2]: https://dnsdist.org/changelog.html#change-1.7.0-alpha2
[3]: https://github.com/PowerDNS/pdns/issues/new/choose
[4]:
https://downloads.powerdns.com/releases/dnsdist-1.7.0-alpha2.tar.bz2
[5]:
hhttps://downloads.powerdns.com/releases/dnsdist-1.7.0-alpha2.tar.bz2.sig
[6]: https://repo.powerdns.com

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] please consider generating a hashed one using 'hashPassword()'

[Remi Gacogne via dnsdist]

Hi Denis,

On 10/2/21 22:47, Denis MACHARD via dnsdist wrote:
With the alpha release of dnsdist 1.7.0, we have the new following log 
message during startup:


Passing a plain-text password via the 'password' parameter to 
'setWebserverConfig()' is not advised, please consider generating a 
hashed one using 'hashPassword()' instead.
Passing a plain-text API key via the 'apiKey' parameter to 
'setWebserverConfig()' is not advised, please consider generating a 
hashed one using 'hashPassword()' instead.


I just want to report that this great feature (PR 10157) ismissing in 
the changelog,


You are right, it did not make it into the Upgrade guide either. I'll 
fix that, thanks!


By the way, I made a small to tool to generate the hash password without 
dnsdist, it's can be useful for automated deployment
https://github.com/dmachard/dnsdist-console 



pip  install  dnsdist_console

python3 -c "from dnsdist_console import HashPassword as
H;print(H().generate(\"mysupersecret\"))"

$scrypt$ln=10,p=1,r=8$rY9YB+QnTOkxKOBlNUUYaw==$4C4Hm5IFiOTluLkjGtPMl4FtYQIwJvSA/eb7uqAlFg4=


That's great, thanks a lot!

Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] DoH pool servers and packet cache usage

[Remi Gacogne via dnsdist]

Hi Denis,

On 10/1/21 09:07, dmachard via dnsdist wrote:

I am trying to use DoH backends the new alpha release of dnsdist 1.7.0

I had a behavior I don’t understand with the packet cache, here the 
configuration used:

[...]
With this configuration, I don’t succeed to use the packet cache with 
incoming UDP dns queries, in the other hand its ok with TCP dns queries


This is the expected behavior for you or perhaps I missed something?


It's indeed a bug, that should be fixed by [1]. Thanks a lot for testing 
the alpha and reporting this!


[1]: https://github.com/PowerDNS/pdns/pull/10787

Best regards,
--
Remi


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] [EXT] Re: Dynamic rule NXDomain

[Remi Gacogne via dnsdist]

Hi John,

Please keep the discussion on the list, so it can benefit others. 
Response inline below.


On 9/23/21 22:46, John Littlekate wrote:

Your explanation is nice and clear, thank you. I have deleted
"QueryRate" rule from dnsdist config for test purposes, restarted
dnsdist daemon and there is a problem, that no rule is applied at all
for that problem client even "DNSRCode.NXDOMAIN" rule is triggered. I
see no reason why that problem client should not be trapped by rule. 
The "DNSRCode.NXDOMAIN" rule works fine, because other client is 
sometimes trapped by this rule.


a/n/au a/ the number of answer records /n/ the number of name server
records /au the number of additional records

I've done some tests to find what is a problem.

If I have send queries by 'dig', rule for NXDomain rate worked well. 
$ dig -t a i...@somedomain.com @10.53.53.153


If I have send same queries by 'host', rule for NXDomain rate did not
work. $ host -t a i...@somedomain.com 10.53.53.153

The difference is only "0/1/0 vs 0/1/1" in resposes.

Most probably, there is a problem with EDNS in responses in rule 
NXDomain rate. If I run, $ dig +noedns -t a i...@somedomain.com

@10.53.53.153 the answer is same as of 'host' answer and NXDomain
rate rule does not work.

Same behaviour is for the DNSRCode.SERVFAIL dynamic rule. I think,
this is a bug.


I tried your dynamic block configuration on the latest master and I 
don't see any issue, the offending client is quickly blocked even when 
the queries are sent via dig +noedns. That's not a big surprise to me 
since we don't parse the content of the response in the dynamic block 
code, we only look at the response code, so I guess there is something 
else at play here. Would you mind providing your full configuration?


Please we aware that we need to have enough queries and responses in our 
ring buffers to get consistent results, at described by Denis Machard in 
[1]. Perhaps that might be your issue?


[1]: 
https://mailman.powerdns.com/pipermail/dnsdist/2021-September/00.html


Best regards,
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: [dnsdist] print the ip address that match a Packet Actions

[Remi Gacogne via dnsdist]

Hi Antoine,

On 9/27/21 14:00, antoine blin via dnsdist wrote:
I'm using the rule : "addAction(MaxQPSIPRule(5), DropAction())" and I'm 
wondering if it is possible to see, through the console API or other 
API, the list of subnet in which rate limit rule is applied.


Not directly, I'm afraid, but you could work something out by setting a 
tag when that rule matches, then trigger a LogAction [1] and finally a 
DropAction when the tag is set. Something like (untested, but you should 
get the idea):


addAction(MaxQPSIPRule(5), SetTagAction("max-qpsip-rule", "match"))
addAction(TagRule("max-qpsip-rule", "match"), 
LogAction('/path/to/dnsdist.log'))

addAction(TagRule("max-qpsip-rule", "match"), DropAction())

Note that this works because LogAction does not stop the processing of 
subsequent rules, as most actions do.


[1]: https://dnsdist.org/rules-actions.html#LogAction

Best regards
--
Remi


OpenPGP_signature
Description: OpenPGP digital signature
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist