Hi all,
I have started replacing my dnsdist servers that are running Debian
Stretch with the updated Debian Buster release. I came across an issue
where dnsdist is running for a period of time it will stop answering any
UDP queries until it is restarted. Sending queries either externally to
the server or from the server itself results in a timeout.
The server is running on bare metal and is receiving around 900-1200
queries/second. There is multiple dnsdist instances running for
different purposes, there is 8 instances total. Each instance is bound
to a different set of IP's, no IP's are shared between dnsdist
instances. For better performance I am using reusePort with 4 listeners
normally, for this issue I tried removing the additional listeners and
removing the reusePort setting but it still occured.
With reusePort enabled the issue occurs after a short period of time
usually between 30 and 40 seconds from the period queries start getting
sent to it. Without reusePort enabled and a single listen thread the
issue seems to occur after around 4-5 hours, when it occurs the symptom
seems the same. Once the issue occurs it doesn't seem to recover itself
(I waited for around 30 hours) until the dnsdist service is restarted.
The dnsdist console and API/web interface continues to work. I can also
see dnsdist sending out the health checking queries to the backends and
it correctly marks them as up or down. TCP queries also work fine.
I had a look through the servers logs but couldn't find anything that
could be related to this problem in syslog/messages/dmesg. I do raise
the open file limit and the tasksmax limits applied by systemd.
The configuration I am using has been migrated from the older 1.3
dnsdist release with a few changes to the configuration to make use of
newer features as the config was built a few years ago.
I also gave the current master release a go but that also had the same
problem.
I am wondering if anyone else has had the same issue? If the
configuration I am using would be useful I can upload a copy of that.
Thanks
_______________________________________________
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist
