Re: [dnsdist] clearing counters?
On Thu, Jul 11, 2019 at 12:11:44PM -0400, Jonathan Reed wrote: > Is there a way to clear counters for all statistics without restarting the > server? Not that I know of. Some counters are actually calculated so it isn't straightforward for us to zero them either. Sorry! Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] Number of Actions
On Mon, Apr 15, 2019 at 08:39:30PM +0200, Klaus Darilion wrote:
> Hello!
>
> Is there a max number of actions, before there might be performance
> problems?
Yes. The design goal is not to have hundreds of rules, but to have fewer,
more powerful rules.
For example, in your case, you can do:
addAction({"lots", "of", "domains"}, DropAction())
This is then fast, much faster than three separate rules.
You can also create a SuffixMatchNode and fill it programatically and then
use a SuffixMatchNodeRule(smn) on it.
Finally for your case, which I spotted on another list, you may want to look
at https://dnsdist.org/guides/dynblocks.html which can be a lot of fun.
Bert
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] dnsdist capacity
> How many the same backend servers I can use (your reccomendation for a lot of > qps)? > > Could you explain more information about the 'sockets' parameter on the > 'newServer()' . Is it mean 'reuseport'? > Thank you for your support! "sockets" increases the number of connections to the same backend: sockets=NUM-- Number of sockets (and thus source ports) used toward the backend server, defaults to a single one From https://dnsdist.org/reference/config.html?highlight=sockets Try 8 and see what happens. Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] can dns responses be changed or suppressed ( E.g. suppress A records for all internal IPs) ?
On Mon, Jun 11, 2018 at 10:09:55AM +0200, Remi Gacogne wrote: > > In short: I'd like to suppress e.g. any A record that contains internal > > IPs from the private IP space in order to hide internal topology better > > from outside clients without the need to run and maintain multiple > > instances of a single dns-zone for internal or external "view". (...) > I'm afraid not. We try hard to limit the parsing of responses to a > minimum in dnsdist for performance reasons, and we don't have the However, it is possible to do this in the PowerDNS Recursor in postresolve(), where records can be stripped or replaced. Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] Cache, chrome and dns tunneling
On Thu, May 03, 2018 at 12:25:28PM -0300, Nico wrote: > We have a bunch of resolvers (unbound and pdns resolver). > And cache, because mobile users only ask for google, facebook and twitter > :-) > cache = newPacketCache(100, 86400, 0, 60, 60) Hi Nico - thanks for sharing this insight! I wonder about a few things. Usually DNS tunnel packets have pretty low TTL and get cleaned quickly enough. In other words, this should not be causing too many problems. Do you know anything about the TTL? > After some tcpdumping and testing we found that chrome and dns tunneling > were filing the cache, > even if the percent of this queries was very low in the total. Chrome is known to generate random queries which get pretty low TTL answers. So again, I wonder if we have a bug somehow. Also, Chrome normally does only a few of these queries, so it is somewhat of a mystery how you end up with so many in the cache. Your cache is limited at 1 million, you could try a bit more. You could also explore the settings of newPacketCache in terms of TTL limits. > Hope it help someone. > (we will be upgrading to 1.3 =very soon) If you could check how 1.3 survives without your special rules, that would be very useful to know. Again, thanks for sharing your results! Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] Latency value under showServers()
On Mon, Mar 19, 2018 at 02:42:54PM +, Nigel H wrote:
> dnsdist -e "grepq('.')" | egrep -v '| internal domain>|arpa'
Best thing you can do is open a ticket asking us to expose grepq via the
API. This would get you this as JSON.
https://github.com/PowerDNS/pdns/issues/new
Thanks!
>
>
> Nigel.
>
> On 19 March 2018 at 11:39, bert hubert <bert.hub...@powerdns.com> wrote:
>
> > On Mon, Mar 19, 2018 at 11:30:08AM +, Nigel H wrote:
> > > I'm seeing a discrepancy when i run showServers() in dnsdist. Our
> > internal
> > > pool of dns servers appears correct and monitoring shows the odd spike in
> > > latency but our external pool (opendns) seems to just keep increasing in
> > > value. The documentation states its the latency for this servers in
> > > milliseconds but when I test using dig I'm seeing normal resolution times
> > > 1-8ms mostly but dnsdist reports latency like below (27ms).
> >
> > Hi Nigel,
> >
> > The likely reason for the increase is that opendns has only received 169
> > and
> > 148 queries so far. This means the weighted average latency still includes
> > a lot of 0.
> >
> > Eventually, you should see the opendns latency plateau.
> >
> > With 'grepq()' you should be able to find the queries being sent to
> > opendns,
> > and how long dnsdist thought they were taking.
> >
> > You could then perhaps retry those queries with 'dig' to see how long they
> > take. 27ms is not unreasonable for mostly cache misses.
> >
> > > Can someone confirm its meaning? Is it an average latency based an
> > amount
> > > of queries to this server or is it the highest latency seen? Also does
> > > dnsdist incorporate the latency of the health check?
> >
> > The last part, no. First part, it is average.
> >
> > Bert
> >
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] Latency value under showServers()
On Mon, Mar 19, 2018 at 11:30:08AM +, Nigel H wrote: > I'm seeing a discrepancy when i run showServers() in dnsdist. Our internal > pool of dns servers appears correct and monitoring shows the odd spike in > latency but our external pool (opendns) seems to just keep increasing in > value. The documentation states its the latency for this servers in > milliseconds but when I test using dig I'm seeing normal resolution times > 1-8ms mostly but dnsdist reports latency like below (27ms). Hi Nigel, The likely reason for the increase is that opendns has only received 169 and 148 queries so far. This means the weighted average latency still includes a lot of 0. Eventually, you should see the opendns latency plateau. With 'grepq()' you should be able to find the queries being sent to opendns, and how long dnsdist thought they were taking. You could then perhaps retry those queries with 'dig' to see how long they take. 27ms is not unreasonable for mostly cache misses. > Can someone confirm its meaning? Is it an average latency based an amount > of queries to this server or is it the highest latency seen? Also does > dnsdist incorporate the latency of the health check? The last part, no. First part, it is average. Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
[dnsdist] Meltdown impact on PowerDNS/dnsdist
Hi everybody, We have done some very tentative measurements on the Linux Meltdown workaround & impact on DNS performance. Based on very early measurements we see around a 10% impact in queries per second for a UDP heavy workload. In addition, one largescale user of PowerDNS Authoritative Server on PostgreSQL suspects the performance problems they see coincided with the rollout of Meltdown workarounds, but we're still investigating. Finally, we did a writeup what Meltdown and specifically Spectre actually are, which you can find on https://ds9a.nl/articles/posts/spectre-meltdown/ We will keep you posted as we learn more! Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
[dnsdist] PowerDNS End-of-Year post
HTML version with clickable links: https://blog.powerdns.com/2017/12/29/powerdns-end-of-year-post-thank-you/ Greetings! 2017 has been a great year for PowerDNS and Open-Xchange. In this post, we want to thank everyone that contributed, and highlight some specific things we are happy about. * HackerOne bug bounty program After some initial problems with over-reporting of non-issues, our experience with HackerOne is awesome right now. We are very happy we have a clean process for receiving and rewarding security bugs. Various PowerDNS security releases this year have originated as HackerOne reports. * Our community PowerDNS continues to be a vibrant community. Our IRC channel has around 240 members, our mailing lists have 1225 subscribers. Even though we are now tougher in enforcing our ‘support, out in the open‘ policies, we continue to see many user queries being resolved every day, often leading to improvements in PowerDNS. As in earlier years, 2017 has seen huge contributions from the community, not only in terms of small patches or constructive bug reports, but also in the revamping of whole subsystems. Specifically Kees Monshouwer was so important for Authoritative Server 4.1 that we would not have been able to do it without him. We hope to continue as a healthy community in 2018! * Facebook bug bounty program PowerDNS is an active participant in keeping the internet secure. As part of our work we found a potential security problem in an important Facebook product which we reported to the their bug bounty program. The bug was fixed quickly, and led to an award of $1500, with the option to turn that into a $3000 charitable donation. We have done so and supported Doctors without Borders in their work. * Our Open Source DNS friends The DNS community is tight, and it has to be: all our software has to interoperate. New standards are developed cooperatively and problems are discussed together. We love the friendly competition that we have with our friends of CZNIC (Knot, Knot Resolver), ISC (BIND), NLNetLabs (NSD, Unbound, libraries) and others. To a huge extent, DNS is exclusively Open Source software, sometimes repackaged and rebadged by commercial companies that close down that Open Source software again. PowerDNS is proud to be part of the open DNS community, and we are grateful for the smooth & fun cooperation we experienced in 2017! * Open-Xchange Since 2015, PowerDNS has been part of Open-Xchange, previously mostly known for the OX AppSuite email platform. The famous Dovecot IMAP project also joined Open-Xchange in 2015. The goal of these mergers was to allow us to focus on technology, while getting the legal, sales and marketing support to get our software out there. In 2017 we have truly started to harvest the fruits of the merger, by simultaneously delivering important software releases as well as satisfying the needs of some very large new deployments. We are very happy that PowerDNS not only survived the merger, but is now an important part of Open-Xchange, where we contribute to the mission of keeping the internet open. * Our users Even without or before contributing codes, operators can improve PowerDNS through great bug reports. We specifically want to thank Quad9 (a collaboration of Packet Clearing House, IBM and the Global Cyber Alliance) for taking a year long journey with us with dnsdist and Recursor “straight from GitHub”. Deployments sharing their experiences and problems with the PowerDNS community are vital to creating quality reliable software. Thanks! * Mattermost, the Open Source private Slack Alternative As PowerDNS grows, we could no longer rely solely on IRC as our communication channel with developers, users and customers. Instead of moving to a third party cloud service that admits to datamining communications, we are very happy to host our own Mattermost instance. And because of PowerDNS user & contributor @42Wim, we can continue our IRC habit with matterircd * 4.1 evolution, dnsdist In 2016 we released the 4.0 versions of the PowerDNS Authoritative Server and Recursor. As you may recall, the 4.0 releases represented a giant cleanup from the decade old frameworks found in 3.x. The 4.0 versions were a step ahead in functionality and sometimes performance, but the true gains of the new fresher codebase have now been realized in the 4.1 releases. 4.1 represents a big overhaul in caching (both Recursor and Authoritative) and DNSSEC processing (mostly Recursor). Both of these overhauls have been tested over the year by large PowerDNS deployments, and the huge amount of feedback has delivered a near flawless “battle tested” 4.1 release. Specifically xs4all and two huge European incumbent operators have been instrumental in maturing dnsdist and our 4.1-era DNSSEC and EDNS Client Subnet implementations. * On to 2018! In 2018 we hope to continue to improve our software and the state of the internet. See you there!
[dnsdist] query multiplication for enhanced reliability, response times
Hello everyone, As the dnsdist installed base grows (a recent large addition is 'quad9.net', https://arstechnica.com/information-technology/2017/11/new-quad9-dns-service-blocks-malicious-domains-for-everyone/ ) we are getting more and more requests for something we previously did not want to do: query multiplication. dnsdist currently takes 1 incoming query packet and hands it to 1 backend. It will not repeat queries if the response is slow or not forthcoming. Various deployments however have backends with suboptimal networks, or want to improve response times by opportunistically asking several backends. An idea that has also been raised is to utilize 'cache snooping' (rd=0) for this purpose. With this message, we invite everyone to share their requirements for query multiplication. We are attempting to deliver something in quite short order, but we'd like to make sure we actually develop the functionality that people need. If you have needs, please share them here. Should you feel uncomfortable doing so in public, you can also email powerdns.id...@powerdns.com off list and we can include your requirements anonymously. This does not apply to support by the way - our procedures for community support are on https://blog.powerdns.com/2016/01/18/open-source-support-out-in-the-open/ Thanks! Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] showConfig()?
On Fri, Oct 06, 2017 at 04:12:51PM +, Summers, William wrote: > I'm testing dnsdist in preproduction, and I am really impressed, thank you > for all the work. Thank you! > I'm wondering if there is a showConfig() style command- similar to a > network device that will dump the current operating configuration of > dnsdist. Seems especially useful to get a global view of operations > especially since they may differ from the config file. Yes, this would indeed be super useful. It also was somewhat of a design goal for dnsdist to have this capability, but eventually we had to give up on it. Your very good question actually asks for 'configuration serialization', which is tricky given how dynamic you can make dnsdist. For example, if you define a Lua function, we can't "get" that Lua function out of Lua anymore. It has been compiled. Interestingly, on https://kolmo.org/ and https://www.youtube.com/watch?v=pb4X-Jjd1yw you can find thoughts on how we could improve this, but we are not there yet. > ?Is there a way to run showServers() at initial console login? Perhaps we could emulate this limited serialization capability. But for now, try delta(). It may do what you want. Bert ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] modify response ip address
On Wed, Aug 02, 2017 at 12:19:35AM +0300, Александр wrote: > Hello everyone. > I've read documentation, but, not being very familiar with lua, didn't get > some parts of it. And didn't find examples in documentation about modifying > response packets based on their contents. This is correct. For now, dnsdist does not support modifying answer packets based on the answer contents. This may change, but for now this is how it is. However, our other product, the PowerDNS Recursor, has a postresolve() hook which does exactly what you need. You could also configure the recursor to forward all queries to your target nameserver, and achieve the same thing. Good luck! Bert > > My question is, is it possible with current state of dnsdist to spoof A and > addresses in response packets based solely on A and fields? > > For example: > Dns query for some random domain (domain is really random, so I can't regexp > domain name) returns A x.y.z.[1-12] and ::::[1-12] > (radom ip from 12 possible for each type) > > If response matches those addresses I want them to be replaced with some > other addresses q.p.y.[1-12] > and ::::[1-12] (also randomly from 12 possible for each > type) > > If it's possible would someone mind to give at least a basic snippet of > code, that may realize this functionality, thanks in advance. > > ___ > dnsdist mailing list > dnsdist@mailman.powerdns.com > https://mailman.powerdns.com/mailman/listinfo/dnsdist ___ dnsdist mailing list dnsdist@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] Trafic redirect via ports
On Thu, Mar 23, 2017 at 09:51:09AM +0100, Thibaud Dublé wrote:
> The first idea was to send traffic to the other dnsdist for "pool1" on port
> 5353 and traffic for "pool2" on port 5354,
> but it looks like it is impossible to add Pool Rules to a specific IP:port.
Hi Thibaud,
I am a bit confused. I think I know what you mean, but perhaps can you
clarify.
> Ideally, the pool rule config would look like this :
>
> bind1NMG = newNMG()
> bind1NMG:addMask("10.255.0.100/32")
> bind1NMG:addMask("10.255.1.100/32")
> bind1NMG:addMask("172.17.0.11:5353")
So you want to add source port selection to the rule?
Are you sure traffic will always come in from that source port? Usually that
is not the case as source ports tend to be dynamic.
If this is what you want, today you could do that with a Lua selection rule:
function luarule(dq)
if(dq.remoteAddr:getPort()==5353)
then
return DNSAction.Pool, "bind"
else
return DNSAction.Pool, "bind2"
end
end
And then:
addLuaAction("172.17.0.11/32", luarule)
You may need to change this a little bit to compensate for typos or errors,
but this is the idea.
Good luck - if this really solves your problem we could make a direct rule
for it. Please let us know.
Bert
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist
Re: [dnsdist] ACL monitoring / Latency
On Mon, Mar 20, 2017 at 09:54:40PM +, Daniel Oakes wrote:
> Hi there,
Morning Daniel, greetings from Monday!
> Just wondering if there's any way of monitoring the ACL Drops at all - I
> couldn't figure out a way.
Not from within dnsdist right now, although it is an interesting idea. What
you can do is make a brief pcap of dnsdist traffic and run 'dnsscope' on it.
It will output a bunch of files one of which is called 'ignores', and this
will be your drops.
> Also we have high latency to our backend PDNS Recursors and we can't work
> out why / how - here's a copy of our config below - we're seeing latency
> of 100+ ms is that normal? For info the 1st two recursors are in the same
> subnet (sub 1ms) and the other two are 16ms:
The best way to debug this is to let your recursor talk to a metronome
server, for example ours. This is explained in
https://blog.powerdns.com/2014/12/11/powerdns-graphing-as-a-service/
This will show the latency distribution and the impact of your network
speeds on that latency.
> pc = newPacketCache(100, 86400, 0, 60, 60)
> getPool(""):setCache(pc)
Ok, this explains a lot. This means only cache -misses- are sent to your
backends. This in turn means latency will always be multiples of your
network latency to the world. From New Zealand, this can be significant.
The reason you get only misses is that all common queries are handled
directly by dnsdist itself.
Good luck!
Bert
___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist
[dnsdist] PowerDNS Year 2016 in review
Hi everyone, We just posted "PowerDNS: 2016 in review" in our blog, https://blog.powerdns.com/2016/12/27/powerdns-2016-in-review/ Text version, which lacks clickable links, is below. But for best results, try the blog! As 2016 draws to a close, we’d like to share a few words on what has been achieved over the past year, our second year within Open-Xchange. This post will cover both our technical and commercial efforts, including the PowerDNS Platform which provides per-subscriber malware filtering & parental control. And, we are hiring! At the end of 2015, we released ‘Technology Preview Releases’ of PowerDNS Authoritative Server 4, PowerDNS Recursor 4 and dnsdist 1.0. This was done to somewhat keep our promise of releasing those versions in 2015, but fell short of what we had hoped to achieve. Now at the end of 2016 the news is a lot better. The actual 4.0 and 1.0 (dnsdist) releases have happened and are being deployed far faster than we’d been hoping for. This is probably due to some of the exciting new features: * RPZ for security & DNS filtering purposes (including IXFR) * dnsdist for reliability, flexibility and DoS protection * pdnsutil edit-zone for a pretty awesome way to edit DNS zones * DNSSEC validation in Recursor * Vastly more powerful Lua engines * ALIAS record type that now powers many of the .GOV search engines DNSSEC (including the White House!) A notable DNSSEC deployment is over at our friends of xs4all who not only sign all their customer domains with the PowerDNS Authoritative Server, but recently have also turned on validation on their PowerDNS Recursors for their large userbase. 4.0 and dnsdist were both part of a ‘spring cleaning’ exercise. It is good to realize how rare it is for a software project to go through such an exercise. 4.0 and dnsdist are based on a much cleaned up and improved codebase. We are also very grateful for our community that stepped up to contribute to 4.x in the form of code, great bug reports, design ideas, documentation and actual bug fixes. Our meagre offering of ‘PowerDNS Crew’ mugs is the least we could do! Some stats that bear out the community involvement: In 2016, our Github repository was forked over a 100 times, yielding almost a 1000 Pull Requests most of which were merged, for a total of over 2500 new commits. These commits closed 1300 issue tickets. As you may recall, since 2015 PowerDNS is part of OX, together with our cousins from Dovecot. When we announced the merger, some voiced fear about what this would mean for PowerDNS. We can now safely say that the state of the PowerDNS source in 2016 is way stronger than it was in 2015. Besides finishing the spring cleaning of our open source products, 2016 also saw the release of the PowerDNS Platform which, unusually for us, is not fully open source. We explained this in our blog post as follows: "Putting it more strongly: we have learned that many organizations simply no longer have the time or desire to assemble all the technologies themselves around our Open Source products. We will therefore be marketing the additional functionalities we have been delivering to our customers as a product tentatively called the “PowerDNS Platform” The “PowerDNS Platform” as we ship it consists of our core unmodified Open Source products, plus loads of other open source technologies, combined with a management shell that is not an Open Source product that we’ll in fact sell." The PowerDNS Platform is described here. Feedback on the move to supply the Platform has been good, both from our commercial users and from the PowerDNS development and wider DNS community, for which we are grateful. Now at the end of 2016 we can report that the PowerDNS Platform has been selected to provide a malware & parental control enabled DNS solution for over 10 million Internet subscribers in Europe. We will be displacing a fully closed solution, which is a win for an open internet. In addition, this commercial progress provides a healthy & sustainable basis on which to continue to develop the PowerDNS nameservers and dnsdist. POWERDNS.ORG We have regained control over powerdns.org. As outlined in our blogpost: "Recently we decided it was time to get the .org back anyhow and after negotiating for a few days we finally paid up, and shortly after that we were back in control of powerdns.org, at a cost of $1000." This personally left me with a bad aftertaste since effectively we have paid a chain of people that specialise in taking over domains for ransom purposes. To compensate for all this, we’ve decided to donate €1000 to the Doctors without Borders charity." MUGS We have shipped close to 500 PowerDNS Release mugs to contributors, friends and conference visitors. If you missed out on our giveaway, you can order PowerDNS mugs online from our friends over at Mugbug, who have been an absolute joy to work with. ROOT-SERVER SPEEDUP We also had a good time working with the fine people of
