Re: BIND Performance with Huge RPZ
> From: Noel Butler > > BIND9 releases can be with the BIND RRL patches by following the link > > labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits > > > > Both of those versions are or will be in official BIND releases. > > I've lost track of which releases have or will have which of those > > two RPZ sets of performance improvements. > > 9.9.4 will have them Not exactly. Checking source finds the min-ns-dots speed-up in 9.8.5-P1, 9.9.3-P1, and 9.9.4b1 and so in 9.9.4. On the other hand, I think the more substantial set of RPZ speed improvements for multiple policy zones is in none of those and so will not be in 9.9.4. My bet would be on 9.10 along with client IP address triggers and "drop" and "truncate" actions. I think the multiple zone speed-up is in the subscription-only 9.9.4-S and so will be in 9.9.4-S1. Vernon Schryverv...@rhyolite.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: BIND Performance with Huge RPZ
Hi, Thanks for the information, really appreciate it, In qps term, each of my server around 10k qps. Currentlt now we are just using simple query logging to file, no syslog yet :( BR, Arie L. Putra 陈维文 -Original Message- From: "Chris Buxton" Sent: 7/12/2013 11:12 PM To: "Arie L.Putra" Cc: "bind-users@lists.isc.org" Subject: Re: BIND Performance with Huge RPZ On Jul 12, 2013, at 3:11 AM, Arie L. Putra wrote: > We are building a server for recursive DNS Server, this server will be acted > as a cache for our network. (several user-side DNS Server will forward to > this server) > Using Ubuntu Server with latest BIND version, we are trying to have RPZ > incuded in this BIND, with around 800k blacklisted sites. > > Has anyone have experience, how RPZ with huge list will impact BIND > performance, will it reduce DNS response time? we have six DNS server that > will point to this server, each server is serving about 15Mbps of DNS Traffic > on peak hour. > > this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), > 16GB RAM. I've seen well over 1 million entries in an RPZ. The performance impact with BIND 9.8 was noticeable but not horrible. The memory requirements were roughly 300 MB for this one zone, compared to over 3 GB for the equivalent in the form of somewhere north of 500 thousand individual zones (two A records each, for the zone apex and a wildcard, all loading from the same file). I'm not used to considering DNS traffic in terms of Mb/s (nor MB/s). I'm more used to considering q/s. The servers with the aforementioned RPZ each handled a relatively large number of queries, possibly as high as 20Kq/s. In my experience, it's impossible to know how a given server will perform without seeing all of the configuration, as lots of configuration settings can impact performance. Once such example is query logging to file (instead of to syslog), which can completely gut performance. Regards, Chris Buxton BLUECAT___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: BIND Performance with Huge RPZ
On Fri, 2013-07-12 at 16:31 +, Vernon Schryver wrote: > Patches for both of those versions of RPZ speed improvements for some > BIND9 releases can be with the BIND RRL patches by following the link > labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits > > Both of those versions are or will be in official BIND releases. > I've lost track of which releases have or will have which of those > two RPZ sets of performance improvements. > 9.9.4 will have them signature.asc Description: This is a digitally signed message part ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: BIND Performance with Huge RPZ
> From: Steven Carr > It's very difficult to predict the impact on performance in general. Yes, the reasonable tactic is to build a representative list of queries from your query logs and use queryperf to hit a test server with those 800K policy zone labels. > But > every query will hit the RPZ zone to be checked against, so make sure you > have enough RAM to hold all 800k records in memory then that will > significantly speed things up. RAM for all 800K records would be sufficient, but might not unnecessary. Names in policy zones are kept in the same kind of red/black trees that are used for ordinary DNS zones, because policy zones are ordinary BIND9 DNS zones. Policy zone IP addresses are kept in radix or patricia trees. Sufficent RAM is the size of the working set. For example, if you have 800K labels below obscure.example.com and almost never resolve any of them, then keeping them in RAM would not help your server's average performance. > The guidance figures that I've seen banded around by a BIND based DNS > appliance is that it will have ~30% impact on query performance per RPZ > feed that has to be looked up against. RPZ performance with zones with NSIP or NSDNAME policies is significantly improved in versions of the RPZ code with the new min-ns-dots parameter with a default value 1. This turns off checking the many server names and IP addresses of TLDs. The version after that has min-ns-dots and also no longer hits every policy zone with every IP address and name in every response, but instead hits a summary red/black tree of names or radix tree of IP addresses summarizing all policy zones once for each address and name. This radically improves RPZ performance with multiple policy zones. The BIND9 tests in bin/tests/system/rpz include a quick and dirty performance test to ensure that nothing is not unexpectedly broken. This is typical output from that test: I:checking performance with RPZ I:checking performance without RPZ I:17758 qps with RPZ is 75% of 23596 qps without RPZ Previous versions of that test did not turn off some very expensive run time checks and so produced much smaller numbers for both cases. Patches for both of those versions of RPZ speed improvements for some BIND9 releases can be with the BIND RRL patches by following the link labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits Both of those versions are or will be in official BIND releases. I've lost track of which releases have or will have which of those two RPZ sets of performance improvements. Vernon Schryverv...@rhyolite.com ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: BIND Performance with Huge RPZ
On Jul 12, 2013, at 3:11 AM, Arie L. Putra wrote: > We are building a server for recursive DNS Server, this server will be acted > as a cache for our network. (several user-side DNS Server will forward to > this server) > Using Ubuntu Server with latest BIND version, we are trying to have RPZ > incuded in this BIND, with around 800k blacklisted sites. > > Has anyone have experience, how RPZ with huge list will impact BIND > performance, will it reduce DNS response time? we have six DNS server that > will point to this server, each server is serving about 15Mbps of DNS Traffic > on peak hour. > > this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), > 16GB RAM. I've seen well over 1 million entries in an RPZ. The performance impact with BIND 9.8 was noticeable but not horrible. The memory requirements were roughly 300 MB for this one zone, compared to over 3 GB for the equivalent in the form of somewhere north of 500 thousand individual zones (two A records each, for the zone apex and a wildcard, all loading from the same file). I'm not used to considering DNS traffic in terms of Mb/s (nor MB/s). I'm more used to considering q/s. The servers with the aforementioned RPZ each handled a relatively large number of queries, possibly as high as 20Kq/s. In my experience, it's impossible to know how a given server will perform without seeing all of the configuration, as lots of configuration settings can impact performance. Once such example is query logging to file (instead of to syslog), which can completely gut performance. Regards, Chris Buxton BLUECAT ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: BIND Performance with Huge RPZ
Hi there, On Fri, 12 Jul 2013, Arie L. Putra wrote: We are building a server for recursive DNS Server, this server will be acted as a cache for our network. (several user-side DNS Server will forward to this server) Using Ubuntu Server with latest BIND version, we are trying to have RPZ incuded in this BIND, with around 800k blacklisted sites. Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour. I wonder if you've considered using iptables in addition to BIND/RPZ? Using the ipsets extension to iptables, on very modest hardware, we routinely block over one thousand million IP addresses with negligible impact on performance. I understand that it's not the same thing at all, but I still wonder if it might be of some use to you. -- 73, Ged. ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: BIND Performance with Huge RPZ
On 12/07/13 11:11, Arie L. Putra wrote: Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour. We don't have that kind of load, but we do have a large (~550k) RPZ setup. It doesn't seem to have any noticeable performance impact, although I should note we're running bind 9.9.2 with the RRL+RPZ patches, which contain some RPZ performance improvements in certain configs: http://ss.vix.su/~vjs/rrlrpz.html Our query load is in the 400-800qps range, with occasional spikes to >1500qps. I had a few problems with RPZ in the past, and it was suggested that our using bind 9.8 (at the time) might have been an issue; we never determined the exact cause, but they don't seem to have recurred on 9.9. ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: BIND Performance with Huge RPZ
On 12 July 2013 11:11, Arie L. Putra wrote: > > Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour. > > this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), 16GB RAM. It's very difficult to predict the impact on performance in general. But every query will hit the RPZ zone to be checked against, so make sure you have enough RAM to hold all 800k records in memory then that will significantly speed things up. The guidance figures that I've seen banded around by a BIND based DNS appliance is that it will have ~30% impact on query performance per RPZ feed that has to be looked up against. Steve ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
BIND Performance with Huge RPZ
Dear All, We are building a server for recursive DNS Server, this server will be acted as a cache for our network. (several user-side DNS Server will forward to this server) Using Ubuntu Server with latest BIND version, we are trying to have RPZ incuded in this BIND, with around 800k blacklisted sites. Has anyone have experience, how RPZ with huge list will impact BIND performance, will it reduce DNS response time? we have six DNS server that will point to this server, each server is serving about 15Mbps of DNS Traffic on peak hour. this server is a Ubuntu box with 2 Xeon (total of 12 core, 24 if include HT), 16GB RAM. Appreciate any information given. Best Regards, Arie Lendra Putra ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
