Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
On Mon, 14 Sep 2020, Mark Andrews wrote: [...] All the queries to the recursive server with this configuration not answered by the server will leak. The configuration needs “forward only;” to be added to prevent the leak. We see this all the time. zone “non-existant-tld” { type forward; forwarders { ; }; forward only; }; Worth making note of! :-) Remember forwarding started off as a performance measure. Falling back to talking to the root servers is desired in that scenario. -- Fred Morris___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
> On 13 Sep 2020, at 06:45, Fred Morris wrote: > > I'm a little confused by the "sky is falling" tone of all this. I'm > pretty sure that the browser fetish for "search and URLs in the same > box!" is pretty high up on the root server pain scale; on any given day > .belkin/.cisco can be one of the top 10 NXDOMAIN TLDs (at least as of a > couple of years ago); search lists are another source of noise (which > doesn't always make it to the roots). > > It's not just quantity, there is also leakage occurring. My recent brain > fart regarding how to turn off the real queries in BIND when something > (typically generated because of a search list) is caught by an RPZ is a > case in point. (This particular use case is to catch noise from stuff > which could be generated from search lists; it's a good use case IMO for > RPZ to generate NXDOMAIN responses locally.) Or, look in your passive > DNS for 127.0.53.53. > > I use DNS, the technology, as a distributed key/value store > occasionally; there is no reason I should have to point it at a tree > with ICANN's roots whatsoever, but for applications which need to be > connected to ICANN's world that can be the past of least resistance. > > Seems to me that if some datacenter is doing beellions of DNS lookups to > DNS as a K/V store some caching for hits as well as NX must be > occurring, seems insane for that not to be the case. I'm trying to > imagine scenarios where those beellions of queries would leak out to the > root servers, or to any servers and none of them look sane to me > (transport ain't free at scale). My biggest concern for the sane cases > would be information leakage, such as the fingerprinting of machines via > AV software doing hash lookups via The DNS; just an example. Private names leak all the time to the DNS. Take zone “non-existant-tld” { type forward; forwarders { ; }; }; and have the server at stop working or just be overloaded. All the queries to the recursive server with this configuration not answered by the server will leak. The configuration needs “forward only;” to be added to prevent the leak. We see this all the time. zone “non-existant-tld” { type forward; forwarders { ; }; forward only; }; Remember forwarding started off as a performance measure. Falling back to talking to the root servers is desired in that scenario. > There are too many borked configurations to enumerate them all. The most > plausible architecture I come up with, perhaps unsurprisingly the one I > would suggest in the absence of explicit mitigating factors, would be a > caching resolver running on the same switch, if not the same machine, as > the application generating the queries. I might secondary the zone > there. Or I might just set it up as a static-stub or forward. > > What are the possible failure modes in this scenario? The application is > configured to use the wrong DNS server: this is not a simple failure, it > has to be configured to use a resolver or nothing resolves; if it's an > authoritative then queries for what's not in bailiwick there are > (hopefully) refused or referred back to the stub resolver (which won't > follow them because it's RD not recursing itself), all of them, not just > ones from the K/V store and it's going to look really really broken; if > it's 8.8.8.8 then surely gmrgle caches the NX response from the root for > the bogus TLD; if it's the corporate caching resolver the same (plus I > expect the phone to ring). The query generates a legitimate NX response > from the zone: that should be the end of it (that's how DNSBLs work); if > search list processing occurs, I'm wondering how one got configured but > more than that how this solution made it into production (if I was > writing the app, I'd probably eschew reliance on the operating system > stub resolver for this particular use case entirely). The app is > misconfigured not to use the correct domain (for the K/V domain): if > it's the wrong domain (or none at all) it doesn't matter whether the > "correct" domain is legitimate or locally concocted. Any others? > > I just don't see how beellions of queries are going to get directed at > the roots or any auth server, regardless of what the TLD is, or if that > occurs that the choice of TLD mitigates in any fashion whatsoever. > There's always a way to make it happen, I just can't imagine it making > it sanely into production even by accident. (This applies to DLV.ISC.ORG > too, which returns an SOA, but they could make it NX if it suited their > purposes.) Actually we can’t remove the DLV.ISC.ORG zone as there is still traffic. That traffic indicates that there are machines that have DLV.ISC.ORG configured as a trust anchor which means we need to return a matching DNSKEY RRset or cause all DNS resolution on those machines to fail. Machines that are still configured to use DLV talk to the servers about once a hour (negative and DNSKEY response TTLs) and they talk to
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
I'm a little confused by the "sky is falling" tone of all this. I'm pretty sure that the browser fetish for "search and URLs in the same box!" is pretty high up on the root server pain scale; on any given day .belkin/.cisco can be one of the top 10 NXDOMAIN TLDs (at least as of a couple of years ago); search lists are another source of noise (which doesn't always make it to the roots). It's not just quantity, there is also leakage occurring. My recent brain fart regarding how to turn off the real queries in BIND when something (typically generated because of a search list) is caught by an RPZ is a case in point. (This particular use case is to catch noise from stuff which could be generated from search lists; it's a good use case IMO for RPZ to generate NXDOMAIN responses locally.) Or, look in your passive DNS for 127.0.53.53. I use DNS, the technology, as a distributed key/value store occasionally; there is no reason I should have to point it at a tree with ICANN's roots whatsoever, but for applications which need to be connected to ICANN's world that can be the past of least resistance. Seems to me that if some datacenter is doing beellions of DNS lookups to DNS as a K/V store some caching for hits as well as NX must be occurring, seems insane for that not to be the case. I'm trying to imagine scenarios where those beellions of queries would leak out to the root servers, or to any servers and none of them look sane to me (transport ain't free at scale). My biggest concern for the sane cases would be information leakage, such as the fingerprinting of machines via AV software doing hash lookups via The DNS; just an example. There are too many borked configurations to enumerate them all. The most plausible architecture I come up with, perhaps unsurprisingly the one I would suggest in the absence of explicit mitigating factors, would be a caching resolver running on the same switch, if not the same machine, as the application generating the queries. I might secondary the zone there. Or I might just set it up as a static-stub or forward. What are the possible failure modes in this scenario? The application is configured to use the wrong DNS server: this is not a simple failure, it has to be configured to use a resolver or nothing resolves; if it's an authoritative then queries for what's not in bailiwick there are (hopefully) refused or referred back to the stub resolver (which won't follow them because it's RD not recursing itself), all of them, not just ones from the K/V store and it's going to look really really broken; if it's 8.8.8.8 then surely gmrgle caches the NX response from the root for the bogus TLD; if it's the corporate caching resolver the same (plus I expect the phone to ring). The query generates a legitimate NX response from the zone: that should be the end of it (that's how DNSBLs work); if search list processing occurs, I'm wondering how one got configured but more than that how this solution made it into production (if I was writing the app, I'd probably eschew reliance on the operating system stub resolver for this particular use case entirely). The app is misconfigured not to use the correct domain (for the K/V domain): if it's the wrong domain (or none at all) it doesn't matter whether the "correct" domain is legitimate or locally concocted. Any others? I just don't see how beellions of queries are going to get directed at the roots or any auth server, regardless of what the TLD is, or if that occurs that the choice of TLD mitigates in any fashion whatsoever. There's always a way to make it happen, I just can't imagine it making it sanely into production even by accident. (This applies to DLV.ISC.ORG too, which returns an SOA, but they could make it NX if it suited their purposes.) Quizzically... -- Fred Morris On 9/10/20 10:57 PM, Rob McEwen wrote: > Mark, > > You gave me the "let them eat cake" answer I anticipated. Also, this > isn't fixing a problem that my services produce - it is preventing a > problem that a potential MISTAKE from a large customer would cause - > the type of mistake that is inevitable at some point, but likely > short-lived. That's on them, not me. But I can sleep well at night > knowing that such MISuse of my service isn't going to take out an > entire datacenter for hours (with MANY innocent bystanders taken out, > too!) with a DOS attack due to those queries NOT ending with a > valid/public domain name, thus making such an attack impossible. > (again, just referring to our very largest customers' DNSBL queries). > [...] > > On 9/11/2020 1:32 AM, Mark Andrews wrote: >>> On 11 Sep 2020, at 15:04, Rob McEwen wrote: >>> >>> Mark, >>> >>> The whole usage of DNS by the anti-spam industry in our DNSBLs - is >>> somewhat a hack on the DNS system from the start - I guess if you >>> think that is wrong, maybe you should take that up with Paul Vixie? >> And Paul will tell you to use a name you control. We did that with >> DLV.ISC.ORG. We are still
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
> On 11 Sep 2020, at 22:22, Rob McEwen wrote: > > On 9/11/2020 2:46 AM, Mark Andrews wrote: >> validate-except (I typo’d it the second time, unfortunately expect and >> except are both valid words). > > I got so far down the rabbit trail with your other points, somehow I missed > that. Thanks. This should solve my problem! > >> If you actually used a zone names with a DNAME > > Great suggestion! I didn't know about that. > However, since i use CloudFlare' DNS for my authoritative DNS - which is > critical for prevention of DDOS attacks - and they don't actually support > DNAME, my hands are tied. (or so it SEEMS - see my question about a possible > workaround at the end of this email) Cloudflare don’t want to deal with the extra database lookup to see if there is a DNAME and the CNAME synthesis. By rejecting zones with DNAMEs they can get away with this stance. > My actual direct query service involves my own rbldnsd servers in 42 cities > around the world (all hiding behind secret host names that a criminal > couldn't easily find) - and those are pointed to by NS records in my > CloudFlare DNS, so then the actual direct DNS queries, and the vast majority > of my DNS traffic for direct queries to my own DNSBL, goes to those 42 > servers around the world, NOT to CloudFlare - but CloudFlare is the starting > point - the first query goes to CloudFlare, then the DNS server doing the > asking "knows" for a while to use one of my own servers, and not bother > CloudFlare with any more traffic for a while. (again, this is for my direct > query service - for my smaller subscribers - my servers can handle THAT > traffic) > But since CloudFlare is the authoritative server for invaluement.com, that is > where the DNAME you're suggesting would need to be setup. Since they don't > support that, I'm not able to implement that at this time. > SEE: https://community.cloudflare.com/t/dname-records-on-cloudflare/16642/4 > > ...also, them not supporting it - makes me a little nervous about others not > supporting it. But maybe that fear is unreasonable since it is only the > "revolvers" that need this feature, not authoritative-only services? This is > something that DNS caching servers like BIND, have been supporting for > decades, correct? DNAME is 2 decades old (August 1999). It came in between DNSSEC version 2 (RFC 2535, KEY/SIG/NXT) and DNSSEC version 3 (RFC 4033/4034/4035, DNSKEY/RRSIG/NSEC/DS). DNSSEC version 3 requires validators to support DNAME. All versions of BIND 9 have supported DNAME. I can’t remember if we added DNAME support to BIND 8 or not. DNSSEC version 4 added NSEC3 and is backwards compatible with DNSSEC version 3. DNSSEC version 4 is what almost all validators support today. > Please tell don't tell me that only a very recent version of BIND does this > correctly. ;) That would probably kill this idea! > POSSIBLE WORKAROUND?: So assuming that DNAME is widely supported by many DNS > caching servers, old and new... I wonder if I could do something similar to > what I do for my direct query service, using NS records to delegate this to > another BIND DNS server that I would run on my own server - so for > "example.invaluement.com" - I'd create a BIND instance on my own server > hosting "example.invaluement.com" as the authoritative server for that zone, > implementing the DNAME records you suggested. Then put a NS record on my > cloudflare telling the world that THIS server is the authoritative server for > "example.invaluement.com" (with TTL for some hours). Do you think that would > work? Delegating to authoritative servers that support DNAME will work. > -- > Rob McEwen > > https://www.invaluement.com > > +1 (478) 475-9032 > > > -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
On 9/11/2020 2:46 AM, Mark Andrews wrote: validate-except (I typo’d it the second time, unfortunately expect and except are both valid words). I got so far down the rabbit trail with your other points, somehow I missed that. Thanks. This should solve my problem! If you actually used a zone names with a DNAME Great suggestion! I didn't know about that. However, since i use CloudFlare' DNS for my authoritative DNS - which is critical for prevention of DDOS attacks - and they don't actually support DNAME, my hands are tied. (or so it SEEMS - see my question about a possible workaround at the end of this email) My actual direct query service involves my own rbldnsd servers in 42 cities around the world (all hiding behind secret host names that a criminal couldn't easily find) - and those are pointed to by NS records in my CloudFlare DNS, so then the actual direct DNS queries, and the vast majority of my DNS traffic for direct queries to my own DNSBL, goes to those 42 servers around the world, NOT to CloudFlare - but CloudFlare is the starting point - the first query goes to CloudFlare, then the DNS server doing the asking "knows" for a while to use one of my own servers, and not bother CloudFlare with any more traffic for a while. (again, this is for my direct query service - for my smaller subscribers - my servers can handle THAT traffic) But since CloudFlare is the authoritative server for invaluement.com, that is where the DNAME you're suggesting would need to be setup. Since they don't support that, I'm not able to implement that at this time. SEE: https://community.cloudflare.com/t/dname-records-on-cloudflare/16642/4 ...also, them not supporting it - makes me a little nervous about others not supporting it. But maybe that fear is unreasonable since it is only the "revolvers" that need this feature, not authoritative-only services? This is something that DNS caching servers like BIND, have been supporting for decades, correct? Please tell don't tell me that _only_ a very _recent_ version of BIND does this correctly. ;) That would probably kill this idea! *POSSIBLE WORKAROUND?:* So assuming that DNAME is widely supported by many DNS caching servers, old and new... I wonder if I could do something similar to what I do for my direct query service, using NS records to delegate this to another BIND DNS server that I would run on my own server - so for "example.invaluement.com" - I'd create a BIND instance on my own server hosting "example.invaluement.com" as the authoritative server for that zone, implementing the DNAME records you suggested. Then put a NS record on my cloudflare telling the world that THIS server is the authoritative server for "example.invaluement.com" (with TTL for some hours). Do you think that would work? -- Rob McEwen https://www.invaluement.com +1 (478) 475-9032 ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
validate-except (I typo’d it the second time, unfortunately expect and except are both valid words). https://downloads.isc.org/isc/bind9/9.16.6/doc/arm/Bv9ARM.pdf validate-except This specifies a list of domain names at and beneath which DNSSEC validation should not be performed, regardless of the presence of a trust anchor at or above those names. This may be used, for example, when configuring a top-level domain intended only for local use, so that the lack of a secure delegation for that domain in the root zone does not cause validation failures. (This is similar to setting a negative trust anchor except that it is a permanent configuration, whereas negative trust anchors expire and are removed after a set period of time.) > On 11 Sep 2020, at 15:57, Rob McEwen wrote: > > Mark, > > You gave me the "let them eat cake" answer I anticipated. Also, this isn't > fixing a problem that my services produce - it is preventing a problem that a > potential MISTAKE from a large customer would cause - the type of mistake > that is inevitable at some point, but likely short-lived. That's on them, not > me. But I can sleep well at night knowing that such MISuse of my service > isn't going to take out an entire datacenter for hours (with MANY innocent > bystanders taken out, too!) with a DOS attack due to those queries NOT ending > with a valid/public domain name, thus making such an attack impossible. > (again, just referring to our very largest customers' DNSBL queries). The query traffic is still getting answered. It doesn’t magically disappear. Its just a matter of who is paying to answer that traffic. If you actually used a zone names with a DNAME at the apex pointing to EMPTY.AS112.ARPA in the public instance, broken servers would leak a couple of queries until the DNAME is cached then the rest of the traffic would be absorbed by the local resolvers and never leave the machine. e.g. example.invaluement.com. DNAME EMPTY.AS112.ARPA. example.invaluement.com. SOA … example.invaluement.com. NS . EMPTY.AS112.ARPA. returns NXDOMAIN for all names under it and it is a distributed service. Named and other recursive servers instantiate instances of EMPTY.AS112.ARPA automatically. Mark > I did a search for "bind9 validate-expect named.conf" (but not in quotes) - > and shockingly LITTLE came up that specifically references that - pages came > up regarding everything else under the sun involving BIND, but I didn't see > anything specifically about that. Do you have a link for that? I'll try to > research that more to try to figure out what exactly you were suggesting. > > Rob McEwen > > On 9/11/2020 1:32 AM, Mark Andrews wrote: >>> On 11 Sep 2020, at 15:04, Rob McEwen wrote: >>> >>> Mark, >>> >>> The whole usage of DNS by the anti-spam industry in our DNSBLs - is >>> somewhat a hack on the DNS system from the start - I guess if you think >>> that is wrong, maybe you should take that up with Paul Vixie? >> And Paul will tell you to use a name you control. We did that with >> DLV.ISC.ORG. We are still absorbing that traffic despite there being no >> entries in the zone for several years now. We knew we would have to do that >> going in. >> >>> And the whole purpose for MANY of us DNSBLs using ".local" in the first >>> place - was precisely to PREVENT the queries from possibly leaking out of >>> our largest customers LANs - because in many cases, that would an >>> essential denial of service attack against us (and our hosters, etc). For >>> example, some DNSBL customers literally have a billion mailboxes. I have a >>> couple of customers with a few hundreds million mailboxes. I'm pretty sure >>> Spamhaus has a few with a billion. Do you have any idea how many emails are >>> processed per second for a billion mailboxes? (especially mid-morning >>> during a business day!) It's enough to where it takes multiple rbldnsd >>> servers each serving thousands of queries per second. To keep up with that >>> volume, these MUST be locally-hosted rbldsnd servers. In that situation, >>> if/when there is a slight DNS mistake - such as some software update >>> mistakenly rerouting DNS to something like "8.8.8.8" - as OFTEN (stupidly!) >>> happens - and then, in the case of Spamhaus' customers with a billion >>> mailboxes - that traffic will massively hit both Google and "spamhaus.org" >>> DNS servers - or even if the forwarder got deleted mistakenly, the same >>> will still happen for "spamhaus.org" DNS servers. Even if those servers can >>> handle the traffic - it might overwhelm a local router in between, or >>> overwhelm the particular DNS server to which this traffic is assigned. This >>> then turns into a NIGHTMARE DOS attack for such DNSBLs. Therefore, the >>> ENTIRE point of using such zone names (".local", ".dnsbl", etc) internally >>> - is to PREVENT the queries from possibly ever leaving the LAN. That is >>> why, for these largest
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
Mark, You gave me the "let them eat cake" answer I anticipated. Also, this isn't fixing a problem that my services produce - it is preventing a problem that a potential MISTAKE from a large customer would cause - the type of mistake that is inevitable at some point, but likely short-lived. That's on them, not me. But I can sleep well at night knowing that such MISuse of my service isn't going to take out an entire datacenter for hours (with MANY innocent bystanders taken out, too!) with a DOS attack due to those queries NOT ending with a valid/public domain name, thus making such an attack impossible. (again, just referring to our very largest customers' DNSBL queries). I did a search for "bind9 validate-expect named.conf" (but not in quotes) - and shockingly LITTLE came up that specifically references that - pages came up regarding everything else under the sun involving BIND, but I didn't see anything specifically about that. Do you have a link for that? I'll try to research that more to try to figure out what exactly you were suggesting. Rob McEwen On 9/11/2020 1:32 AM, Mark Andrews wrote: On 11 Sep 2020, at 15:04, Rob McEwen wrote: Mark, The whole usage of DNS by the anti-spam industry in our DNSBLs - is somewhat a hack on the DNS system from the start - I guess if you think that is wrong, maybe you should take that up with Paul Vixie? And Paul will tell you to use a name you control. We did that with DLV.ISC.ORG. We are still absorbing that traffic despite there being no entries in the zone for several years now. We knew we would have to do that going in. And the whole purpose for MANY of us DNSBLs using ".local" in the first place - was precisely to PREVENT the queries from possibly leaking out of our largest customers LANs - because in many cases, that would an essential denial of service attack against us (and our hosters, etc). For example, some DNSBL customers literally have a billion mailboxes. I have a couple of customers with a few hundreds million mailboxes. I'm pretty sure Spamhaus has a few with a billion. Do you have any idea how many emails are processed per second for a billion mailboxes? (especially mid-morning during a business day!) It's enough to where it takes multiple rbldnsd servers each serving thousands of queries per second. To keep up with that volume, these MUST be locally-hosted rbldsnd servers. In that situation, if/when there is a slight DNS mistake - such as some software update mistakenly rerouting DNS to something like "8.8.8.8" - as OFTEN (stupidly!) happens - and then, in the case of Spamhaus' customers with a billion mailboxes - that traffic will massively hit both Google and "spamhaus.org" DNS servers - or even if the forwarder got deleted mistakenly, the same will still happen for "spamhaus.org" DNS servers. Even if those servers can handle the traffic - it might overwhelm a local router in between, or overwhelm the particular DNS server to which this traffic is assigned. This then turns into a NIGHTMARE DOS attack for such DNSBLs. Therefore, the ENTIRE point of using such zone names (".local", ".dnsbl", etc) internally - is to PREVENT the queries from possibly ever leaving the LAN. That is why, for these largest customers, using hostnames that end in our own domain names - is not an option. (and when it does work, it is often a "let them eat cake" option - where only the largest Internet companies with billions in revenue - can afford to handle such traffic - so please, don't respond with a "let them eat cake" answer!) But that overall point about how DNSBLs work in such situations... seems lost on you. The very reason I used ".dnsbl" as an example - is because I did a little research after before last email - and it turns out that - maybe in response to the RFC you pointed out that took a position against using ".local" - Spamhaus then (apparently) switched to using ".dnsbl" - (or maybe they were using ".dnsbl" all along? - I can't keep track over every other DNSBL - but ".local" was popular for many DNSBLs for many years.) Spamhaus doesn't use that for their direct query service - but it appears that they're using that for the instructions for their customers who RSYNC the data. Therefore, you just harshly criticized me for suggesting doing what Spamhaus ALREADY does - so I guess I'm in good company! Two wrongs don’t make a right. If you think queries will leak then provision services to absorb those leaks. The root operators shouldn’t have to absorb that traffic. RFC 1918 DNS traffic leaked and services where stood up to absorb that leaking traffic. There is nothing stopping you from doing something similar. Absolutely nothing. Really - your purism - and harsh realities of large DSNBL operations - are not compatible. No, its taking ownership of the problems that your operations produce. You are cost shifting by not doing so. And no - you NEVER gave me an answer - and guess what? While I
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
> On 11 Sep 2020, at 15:04, Rob McEwen wrote: > > Mark, > > The whole usage of DNS by the anti-spam industry in our DNSBLs - is somewhat > a hack on the DNS system from the start - I guess if you think that is wrong, > maybe you should take that up with Paul Vixie? And Paul will tell you to use a name you control. We did that with DLV.ISC.ORG. We are still absorbing that traffic despite there being no entries in the zone for several years now. We knew we would have to do that going in. > And the whole purpose for MANY of us DNSBLs using ".local" in the first place > - was precisely to PREVENT the queries from possibly leaking out of our > largest customers LANs - because in many cases, that would an essential > denial of service attack against us (and our hosters, etc). For example, some > DNSBL customers literally have a billion mailboxes. I have a couple of > customers with a few hundreds million mailboxes. I'm pretty sure Spamhaus has > a few with a billion. Do you have any idea how many emails are processed per > second for a billion mailboxes? (especially mid-morning during a business > day!) It's enough to where it takes multiple rbldnsd servers each serving > thousands of queries per second. To keep up with that volume, these MUST be > locally-hosted rbldsnd servers. In that situation, if/when there is a slight > DNS mistake - such as some software update mistakenly rerouting DNS to > something like "8.8.8.8" - as OFTEN (stupidly!) happens - and then, in the > case of Spamhaus' customers with a billion mailboxes - that traffic will > massively hit both Google and "spamhaus.org" DNS servers - or even if the > forwarder got deleted mistakenly, the same will still happen for > "spamhaus.org" DNS servers. Even if those servers can handle the traffic - it > might overwhelm a local router in between, or overwhelm the particular DNS > server to which this traffic is assigned. This then turns into a NIGHTMARE > DOS attack for such DNSBLs. Therefore, the ENTIRE point of using such zone > names (".local", ".dnsbl", etc) internally - is to PREVENT the queries from > possibly ever leaving the LAN. That is why, for these largest customers, > using hostnames that end in our own domain names - is not an option. (and > when it does work, it is often a "let them eat cake" option - where only the > largest Internet companies with billions in revenue - can afford to handle > such traffic - so please, don't respond with a "let them eat cake" answer!) > But that overall point about how DNSBLs work in such situations... seems lost > on you. > > The very reason I used ".dnsbl" as an example - is because I did a little > research after before last email - and it turns out that - maybe in response > to the RFC you pointed out that took a position against using ".local" - > Spamhaus then (apparently) switched to using ".dnsbl" - (or maybe they were > using ".dnsbl" all along? - I can't keep track over every other DNSBL - but > ".local" was popular for many DNSBLs for many years.) Spamhaus doesn't use > that for their direct query service - but it appears that they're using that > for the instructions for their customers who RSYNC the data. Therefore, you > just harshly criticized me for suggesting doing what Spamhaus ALREADY does - > so I guess I'm in good company! Two wrongs don’t make a right. If you think queries will leak then provision services to absorb those leaks. The root operators shouldn’t have to absorb that traffic. RFC 1918 DNS traffic leaked and services where stood up to absorb that leaking traffic. There is nothing stopping you from doing something similar. Absolutely nothing. > Really - your purism - and harsh realities of large DSNBL operations - are > not compatible. No, its taking ownership of the problems that your operations produce. You are cost shifting by not doing so. > And no - you NEVER gave me an answer - and guess what? While I have > tremendous respect for RFCs in general, and try hard to follow them - they > are NOT perfect - on rare occasion, some of them SHOULD be broken and DO have > errors or situations that they didn't anticipate. This one of those. RFCs > were written by humans. Humans make mistakes Actually I did. I said "add validate-expect entries to named.conf”. > And it's too bad that the maintainers of BIND didn't anticipate that there > might be local-data situations where sys admins should be given the ability > to turn DNSSEC off for a particular zone. Your answers are helping me to > understand HOW/WHY such decisions were made. But rigidity/purity doesn't > always equal wisdom/intelligence. In this case, it doesn't. > > Rob McEwen, invaluement > > On 9/10/2020 10:23 PM, Mark Andrews wrote: >>> On 11 Sep 2020, at 11:13, Rob McEwen wrote: >>> >>> Mark, >>> >>> Most invaluement subscribers do direct queries - to hostnames that end with >>> my own valid domain names that don't have this DNSSEC
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
Mark, The whole usage of DNS by the anti-spam industry in our DNSBLs - is somewhat a hack on the DNS system from the start - I guess if you think that is wrong, maybe you should take that up with Paul Vixie? And the whole purpose for MANY of us DNSBLs using ".local" in the first place - was precisely to PREVENT the queries from possibly leaking out of our largest customers LANs - because in many cases, that would an essential denial of service attack against us (and our hosters, etc). For example, some DNSBL customers literally have a billion mailboxes. I have a couple of customers with a few hundreds million mailboxes. I'm pretty sure Spamhaus has a few with a billion. Do you have any idea how many emails are processed per second for a billion mailboxes? (especially mid-morning during a business day!) It's enough to where it takes multiple rbldnsd servers each serving thousands of queries per second. To keep up with that volume, these MUST be locally-hosted rbldsnd servers. In that situation, if/when there is a slight DNS mistake - such as some software update mistakenly rerouting DNS to something like "8.8.8.8" - as OFTEN (stupidly!) happens - and then, in the case of Spamhaus' customers with a billion mailboxes - that traffic will massively hit both Google and "spamhaus.org" DNS servers - or even if the forwarder got deleted mistakenly, the same will still happen for "spamhaus.org" DNS servers. Even if those servers can handle the traffic - it might overwhelm a local router in between, or overwhelm the particular DNS server to which this traffic is assigned. This then turns into a NIGHTMARE DOS attack for such DNSBLs. Therefore, the ENTIRE point of using such zone names (".local", ".dnsbl", etc) internally - is to PREVENT the queries from possibly ever leaving the LAN. That is why, for these largest customers, using hostnames that end in our own domain names - is not an option. (and when it does work, it is often a "let them eat cake" option - where only the largest Internet companies with billions in revenue - can afford to handle such traffic - so please, don't respond with a "let them eat cake" answer!) But that overall point about how DNSBLs work in such situations... seems lost on you. The very reason I used ".dnsbl" as an example - is because I did a little research after before last email - and it turns out that - maybe in response to the RFC you pointed out that took a position against using ".local" - Spamhaus then (apparently) switched to using ".dnsbl" - (or maybe they were using ".dnsbl" all along? - I can't keep track over every other DNSBL - but ".local" was popular for many DNSBLs for many years.) Spamhaus doesn't use that for their direct query service - but it appears that they're using that for the instructions for their customers who RSYNC the data. Therefore, you just harshly criticized me for suggesting doing what Spamhaus ALREADY does - so I guess I'm in good company! Really - your purism - and harsh realities of large DSNBL operations - are not compatible. And no - you NEVER gave me an answer - and guess what? While I have tremendous respect for RFCs in general, and try hard to follow them - they are NOT perfect - on rare occasion, some of them SHOULD be broken and DO have errors or situations that they didn't anticipate. This one of those. RFCs were written by humans. Humans make mistakes. And it's too bad that the maintainers of BIND didn't anticipate that there might be local-data situations where sys admins should be given the ability to turn DNSSEC off for a particular zone. Your answers are helping me to understand HOW/WHY such decisions were made. But rigidity/purity doesn't always equal wisdom/intelligence. In this case, it doesn't. Rob McEwen, invaluement On 9/10/2020 10:23 PM, Mark Andrews wrote: On 11 Sep 2020, at 11:13, Rob McEwen wrote: Mark, Most invaluement subscribers do direct queries - to hostnames that end with my own valid domain names that don't have this DNSSEC issue - those are the ONE ones that make use of public DNS and are broadcast across the internet. Our usage of ".local" zones for those who are RSYNC'ing our data - dates back to something like 2007, and the RFC you referred to is from 2013. By the time this RFC had been published, we'd already had customer using the ".local" for 6 years. At the time that came out in 2013, I assessed whether I needed to get my clients to change that, but it didn't seem to effect anyone. Again, those of our subscribers who RSYNC our data and use the ".local" zone names - are just using that for 100% local usage, and are not trying to broadcast it across the internet. And in many of THOSE cases, if the BIND and RBLDND are on the same computer, as is often the case, it doesn't even go out to the LAN - this is all on one single computer. And you squatted on .local then and are paying the price now. It has always been wrong to use a name
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
> On 11 Sep 2020, at 11:13, Rob McEwen wrote: > > Mark, > > Most invaluement subscribers do direct queries - to hostnames that end with > my own valid domain names that don't have this DNSSEC issue - those are the > ONE ones that make use of public DNS and are broadcast across the internet. > > Our usage of ".local" zones for those who are RSYNC'ing our data - dates back > to something like 2007, and the RFC you referred to is from 2013. By the time > this RFC had been published, we'd already had customer using the ".local" for > 6 years. At the time that came out in 2013, I assessed whether I needed to > get my clients to change that, but it didn't seem to effect anyone. Again, > those of our subscribers who RSYNC our data and use the ".local" zone names - > are just using that for 100% local usage, and are not trying to broadcast it > across the internet. And in many of THOSE cases, if the BIND and RBLDND are > on the same computer, as is often the case, it doesn't even go out to the LAN > - this is all on one single computer. And you squatted on .local then and are paying the price now. It has always been wrong to use a name that has not been delegated to you. The point of having delegations in the DNS is to prevent multiple entities using the same name and to give certainty to those the name is delegated to. That has been the case since the DNS was developed. As for not leaving the machine, that machine is connected to a network where mDNS may be in use. That creates a namespace collision on that machine. What does invaluement.local mean on the machine? Does the machine use mDNS or DNS to resolve the name? > So are you claiming that if I simply changed the zone naming form ending in > ".local" - to something else - such as ".dnsbl" - then all my problems would > go away? And the forwarder will start working? (even though rbldnsd doesn't > do DNSSEC) No. You have not been delegated ".dnsbl”. IANA/ICANN owns *every* possible tld name not delegated / allocated to someone/something else. Any TLD that you pick will have the same issue. DNSSEC proves what exists, what doesn’t exist, and what isn’t secured by DNSSEC. You have a effectively infinite number of names below invaluement.com. Pick some of them and use them. invaluement.com isn’t signed so your customers won’t have DNSSEC problems. When you decide you want to sign invaluement.com you will need to break the DNSSEC chain of trust by having a delegation in the invaluement.com which doesn’t have a DS record with it. > That would be EXCELLENT news! Or, if that doesn't actually fix my problem, do > you have any suggestions that actually address my actual question? I gave you a answer. See below. Mark > Rob McEwen > > On 9/10/2020 7:37 PM, Mark Andrews wrote: >> .local is for mDNS (RFC 6762). Do not use it for other purposes as you are >> hijacking the namespace. >> >> The best solution is to NOT change the name of the zones from those that you >> use publicly. That way they have the correct DNSSEC chain of trust down >> from the root. If you want to use different zone names then create >> delegations to empty unsigned zones (SOA and NS records only) like those >> done for 10.IN-ADDR.ARPA in a zone you control. That breaks the DNSSEC >> chain of trust at the delegation point. If you later decide you want to >> sign these zones you can do so and link them into the DNSSEC chain of trust. >> Just sign both the rbldsnd-formatted files and the empty zones. >> >> If you absolutely must continue to hijack the .local namespace, which is >> allocated for a different purpose, then add validate-except entries to >> named.conf >> >> Mark >> >>> On 11 Sep 2020, at 01:56, Rob McEwen wrote: >>> >>> I manage an anti-spam DNSBL and I've been running into an issue in recent >>> years - that I'm FINALLY getting around to asking about. I just joined this >>> list to ask this question. Also, I checked the archives, but couldn't find >>> an answer - at least, not one I understood. >>> >>> So basically, while most of our users do direct queries and don't have this >>> issue - some of our larger subscribers RSYNC the rbldsnd-formatted files, >>> and then they typically run rbldnsd on the same server as their BIND server >>> that is answering their DNSBL queries. Then, their invaluement zone names >>> will all end with "invaluement.local". Typically, their RBLDNSD server is >>> set up to listen on 127.0.0.2 - and then they use BIND for answering their >>> DNSBL queries, and so they tell BIND to get its answers for THOSE >>> invaluement dnsbl queries by doing a DNS forwarder, telling bind to get the >>> answers for THOSE zones from 127.0.0.2 - as shown below: >>> >>> zone "invaluement.local" in { >>> type forward; >>> forward only; >>> forwarders { 127.0.0.2; }; >>> }; >>> >>> This works perfectly - so long as DNSSEC is turned off. And since most of >>> our subscribers are running
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
Mark, Most invaluement subscribers do direct queries - to hostnames that end with my own valid domain names that don't have this DNSSEC issue - those are the ONE ones that make use of public DNS and are broadcast across the internet. Our usage of ".local" zones for those who are RSYNC'ing our data - dates back to something like 2007, and the RFC you referred to is from 2013. By the time this RFC had been published, we'd already had customer using the ".local" for 6 years. At the time that came out in 2013, I assessed whether I needed to get my clients to change that, but it didn't seem to effect anyone. Again, those of our subscribers who RSYNC our data and use the ".local" zone names - are just using that for 100% local usage, and are not trying to broadcast it across the internet. And in many of THOSE cases, if the BIND and RBLDND are on the same computer, as is often the case, it doesn't even go out to the LAN - this is all on one single computer. So are you claiming that if I simply changed the zone naming form ending in ".local" - to something else - such as ".dnsbl" - then all my problems would go away? And the forwarder will start working? (even though rbldnsd doesn't do DNSSEC) That would be EXCELLENT news! Or, if that doesn't actually fix my problem, do you have any suggestions that actually address my actual question? Rob McEwen On 9/10/2020 7:37 PM, Mark Andrews wrote: .local is for mDNS (RFC 6762). Do not use it for other purposes as you are hijacking the namespace. The best solution is to NOT change the name of the zones from those that you use publicly. That way they have the correct DNSSEC chain of trust down from the root. If you want to use different zone names then create delegations to empty unsigned zones (SOA and NS records only) like those done for 10.IN-ADDR.ARPA in a zone you control. That breaks the DNSSEC chain of trust at the delegation point. If you later decide you want to sign these zones you can do so and link them into the DNSSEC chain of trust. Just sign both the rbldsnd-formatted files and the empty zones. If you absolutely must continue to hijack the .local namespace, which is allocated for a different purpose, then add validate-except entries to named.conf Mark On 11 Sep 2020, at 01:56, Rob McEwen wrote: I manage an anti-spam DNSBL and I've been running into an issue in recent years - that I'm FINALLY getting around to asking about. I just joined this list to ask this question. Also, I checked the archives, but couldn't find an answer - at least, not one I understood. So basically, while most of our users do direct queries and don't have this issue - some of our larger subscribers RSYNC the rbldsnd-formatted files, and then they typically run rbldnsd on the same server as their BIND server that is answering their DNSBL queries. Then, their invaluement zone names will all end with "invaluement.local". Typically, their RBLDNSD server is set up to listen on 127.0.0.2 - and then they use BIND for answering their DNSBL queries, and so they tell BIND to get its answers for THOSE invaluement dnsbl queries by doing a DNS forwarder, telling bind to get the answers for THOSE zones from 127.0.0.2 - as shown below: zone "invaluement.local" in { type forward; forward only; forwarders { 127.0.0.2; }; }; This works perfectly - so long as DNSSEC is turned off. And since most of our subscribers are running a dedicated instance of BIND that is ONLY used for DNSBL queries, they don't mind turning DNSSEC off. But, occasionally, we have a customer who cannot turn DNSSEC off. So I was hoping that THIS command would work: dnssec-must-be-secure "invaluement.local" no; But it doesn't seem to be helping at all. Is that command suppose to disable DNSSEC checking for a particular zone? If yes, what did I do wrong? If not, what does "dnssec-must-be-secure" set to "no" do? I've heard that there is NOT a way to get this to work - and that such subscribers much use DNS Delegation, instead. But I really wish this could be done by simply turning off DNSSEC for a particular zone. That could be useful for MANY various types of internal zones that need this. But if this is that case, how would that DNS Delegation look, to get the above forwarding example to work using delegation instead? Thanks in advance for your help! -- Rob McEwen, invaluement ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users -- Rob McEwen https://www.invaluement.com +1 (478) 475-9032 ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
.local is for mDNS (RFC 6762). Do not use it for other purposes as you are hijacking the namespace. The best solution is to NOT change the name of the zones from those that you use publicly. That way they have the correct DNSSEC chain of trust down from the root. If you want to use different zone names then create delegations to empty unsigned zones (SOA and NS records only) like those done for 10.IN-ADDR.ARPA in a zone you control. That breaks the DNSSEC chain of trust at the delegation point. If you later decide you want to sign these zones you can do so and link them into the DNSSEC chain of trust. Just sign both the rbldsnd-formatted files and the empty zones. If you absolutely must continue to hijack the .local namespace, which is allocated for a different purpose, then add validate-except entries to named.conf Mark > On 11 Sep 2020, at 01:56, Rob McEwen wrote: > > I manage an anti-spam DNSBL and I've been running into an issue in recent > years - that I'm FINALLY getting around to asking about. I just joined this > list to ask this question. Also, I checked the archives, but couldn't find an > answer - at least, not one I understood. > > So basically, while most of our users do direct queries and don't have this > issue - some of our larger subscribers RSYNC the rbldsnd-formatted files, and > then they typically run rbldnsd on the same server as their BIND server that > is answering their DNSBL queries. Then, their invaluement zone names will all > end with "invaluement.local". Typically, their RBLDNSD server is set up to > listen on 127.0.0.2 - and then they use BIND for answering their DNSBL > queries, and so they tell BIND to get its answers for THOSE invaluement dnsbl > queries by doing a DNS forwarder, telling bind to get the answers for THOSE > zones from 127.0.0.2 - as shown below: > > zone "invaluement.local" in { > type forward; > forward only; > forwarders { 127.0.0.2; }; > }; > > This works perfectly - so long as DNSSEC is turned off. And since most of our > subscribers are running a dedicated instance of BIND that is ONLY used for > DNSBL queries, they don't mind turning DNSSEC off. > > But, occasionally, we have a customer who cannot turn DNSSEC off. So I was > hoping that THIS command would work: > > dnssec-must-be-secure "invaluement.local" no; > > But it doesn't seem to be helping at all. Is that command suppose to disable > DNSSEC checking for a particular zone? If yes, what did I do wrong? If not, > what does "dnssec-must-be-secure" set to "no" do? > > I've heard that there is NOT a way to get this to work - and that such > subscribers much use DNS Delegation, instead. But I really wish this > could be done by simply turning off DNSSEC for a particular zone. That could > be useful for MANY various types of internal zones that need this. But if > this is that case, how would that DNS Delegation look, to get the above > forwarding example to work using delegation instead? > > Thanks in advance for your help! > > -- > Rob McEwen, invaluement > > > ___ > Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe > from this list > > ISC funds the development of this software with paid support subscriptions. > Contact us at https://www.isc.org/contact/ for more information. > > > bind-users mailing list > bind-users@lists.isc.org > https://lists.isc.org/mailman/listinfo/bind-users -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
On Thu, 2020-09-10 at 13:50 -0400, Jim Popovitch via bind-users wrote: > On Thu, 2020-09-10 at 11:56 -0400, Rob McEwen wrote: > > I manage an anti-spam DNSBL and I've been running into an issue in recent > > years - that I'm FINALLY getting around to asking about. I just joined this > > list to ask this question. Also, I checked the archives, but couldn't find > > an answer - at least, not one I understood. > > So basically, while most of our users do direct queries and don't have this > > issue - some of our larger subscribers RSYNC the rbldsnd-formatted files, > > and then they typically run rbldnsd on the same server as their BIND server > > that is answering their DNSBL queries. Then, their invaluement zone names > > will all end with "invaluement.local". Typically, their RBLDNSD server is > > set up to listen on 127.0.0.2 - and then they use BIND for answering their > > DNSBL queries, and so they tell BIND to get its answers for THOSE > > invaluement dnsbl queries by doing a DNS forwarder, telling bind to get the > > answers for THOSE zones from 127.0.0.2 - as shown below: > > zone "invaluement.local" in { > > type forward; > > forward only; > > forwarders { 127.0.0.2; }; > > }; > > > > This works perfectly - so long as DNSSEC is turned off. And since most of > > our subscribers are running a dedicated instance of BIND that is ONLY used > > for DNSBL queries, they don't mind turning DNSSEC off. > > But, occasionally, we have a customer who cannot turn DNSSEC off. So I was > > hoping that THIS command would work: > > dnssec-must-be-secure "invaluement.local" no; > > But it doesn't seem to be helping at all. Is that command suppose to > > disable DNSSEC checking for a particular zone? If yes, what did I do wrong? > > If not, what does "dnssec-must-be-secure" set to "no" do? > > I've heard that there is NOT a way to get this to work - and that such > > subscribers much use DNS Delegation, instead. But I really wish this could > > be done by simply turning off DNSSEC for a particular zone. That could be > > useful for MANY various types of internal zones that need this. But if this > > is that case, how would that DNS Delegation look, to get the above > > forwarding example to work using delegation instead? > > Thanks in advance for your help! > > Just a thought, but ARM says: > > dnssec-must-be-secure >Specify hierarchies which must be or may not be secure (signed and > validated). If yes, >then named will only accept answers if they are secure. If no, then normal > DNSSEC >validation applies allowing for insecure answers to be accepted. The > specified domain >must be defined as a trust anchor, for instance in a trust-anchors > statement, or dnssec- >validation auto must be active. > > > You might want to try adding "dnssec-validation auto" to the zone stanza. > > zone "invaluement.local" in { >type forward; >forward only; >forwarders { 127.0.0.2; }; >dnssec-validation auto; > }; In retrospect that won't work. dnssec-validation can only be used in options or views. Maybe try using a view? -Jim P. ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: rbldnsd and DNSSEC compatibility issues - any suggestions?
On Thu, 2020-09-10 at 11:56 -0400, Rob McEwen wrote: > I manage an anti-spam DNSBL and I've been running into an issue in recent > years - that I'm FINALLY getting around to asking about. I just joined this > list to ask this question. Also, I checked the archives, but couldn't find an > answer - at least, not one I understood. > So basically, while most of our users do direct queries and don't have this > issue - some of our larger subscribers RSYNC the rbldsnd-formatted files, and > then they typically run rbldnsd on the same server as their BIND server that > is answering their DNSBL queries. Then, their invaluement zone names will all > end with "invaluement.local". Typically, their RBLDNSD server is set up to > listen on 127.0.0.2 - and then they use BIND for answering their DNSBL > queries, and so they tell BIND to get its answers for THOSE invaluement dnsbl > queries by doing a DNS forwarder, telling bind to get the answers for THOSE > zones from 127.0.0.2 - as shown below: > zone "invaluement.local" in { > type forward; > forward only; > forwarders { 127.0.0.2; }; > }; > > This works perfectly - so long as DNSSEC is turned off. And since most of our > subscribers are running a dedicated instance of BIND that is ONLY used for > DNSBL queries, they don't mind turning DNSSEC off. > But, occasionally, we have a customer who cannot turn DNSSEC off. So I was > hoping that THIS command would work: > dnssec-must-be-secure "invaluement.local" no; > But it doesn't seem to be helping at all. Is that command suppose to disable > DNSSEC checking for a particular zone? If yes, what did I do wrong? If not, > what does "dnssec-must-be-secure" set to "no" do? > I've heard that there is NOT a way to get this to work - and that such > subscribers much use DNS Delegation, instead. But I really wish this could be > done by simply turning off DNSSEC for a particular zone. That could be useful > for MANY various types of internal zones that need this. But if this is that > case, how would that DNS Delegation look, to get the above forwarding example > to work using delegation instead? > Thanks in advance for your help! Just a thought, but ARM says: dnssec-must-be-secure Specify hierarchies which must be or may not be secure (signed and validated). If yes, then named will only accept answers if they are secure. If no, then normal DNSSEC validation applies allowing for insecure answers to be accepted. The specified domain must be defined as a trust anchor, for instance in a trust-anchors statement, or dnssec- validation auto must be active. You might want to try adding "dnssec-validation auto" to the zone stanza. zone "invaluement.local" in { type forward; forward only; forwarders { 127.0.0.2; }; dnssec-validation auto; }; -Jim P. ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users