Re: [mailop] Feasibility of a private DNSBL
On 10/11/2021 19:47, Rob McEwen via mailop wrote: > On 11/10/2021 2:31 PM, Chris via mailop wrote: >> like any other DNS server, you *can* implement a single key by putting >> your DNSBL zone under a name like ".example.com" > > > The only issue here is that, for every user/customer that needs a unique > key, an entirely different set of data has to be loaded into memory on > the server. That's a huge limitation. It doesn't "scale". Therefore, for > invaluement, in our new-ish direct query system (that started in 2018), > that uses such unique keys for each DNS query customer, I had to > basically custom program rbldnsd to overcome this. It took dozens of > hours of very frustratingly-difficult programing, but that's partly > because I'm not very good at C++! (by doing this myself, that was > especially helpful for keeping this very hard-earned expertise in-house > and very guarded!) > > I'm not sure how the Rspamd version handles that memory issue? But for > what I do, there's just one dataset in memory for each zone that serves > queries that have _different_ license keys in the license key DNS-label > part (but where the zone name is /otherwise/ the same). So this is very > efficient! My version of rbldnsd can apparently handle it by adding a new service zone type (like acl zone): https://github.com/rspamd/rbldnsd/blob/master/rbldnsd_aclkey.c Internally, it has a hash table of key/reply entries which can be reloaded and attached to any other zone. It also removes the key from the DNS query, so the underlying zone can deal with the rest. In general, it works almost exactly the same as the original acl set. I'm also thinking about adding DNSCurve encryption there (up to the RBL address labels), as it has almost zero cost and is already supported by Rspamd. This will eliminate lot's of privacy issues when you think about putting of the sensible data in DNS (even access keys). ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 11/10/2021 9:58 PM, Michael Peddemors via mailop wrote: On 2021-11-10 11:47 a.m., Rob McEwen via mailop wrote: The only issue here is that, for every user/customer that needs a unique key, an entirely different set of data has to be loaded into memory on the server. That's a huge limitation. It doesn't "scale". Therefore, for invaluement, in our new-ish direct query system (that started in 2018), that uses such unique keys for each DNS query customer, I had to basically custom program rbldnsd to overcome this. It took dozens of hours of very frustratingly-difficult programing, but that's partly because I'm not very good at C++! (by doing this myself, that was especially helpful for keeping this very hard-earned expertise in-house and very guarded!) Not sure why we/you should keep it 'inhouse', as it seems that all RBL operators are adopting a similar strategy. We need more co-operative opensource collaboration, and ensure that RBLDNSD doesn't end up with twenty forks, so it gets even better performance, and simply has an option to treat all queries to a 'keyed' subdomain, either to be served a single data source, or individual data sources, based on a configuration item ;) And of course, the DSO support ;) Already, the source/git are in too many multiple places. Whatever happened to giving back to opensource? I think you are building on other's 'hard-earned expertise' as well. (1) I give back in other ways - some you don't know about - but, coincidentally - there are some interesting upcoming announcements about how/what invaluement is giving back - coming soon - I'll be sure to send you an alert about those upcoming announcements when they happen! (2) interesting how this argument always gets applied to entrepreneurs who aren't at all wealthy, but rarely to BigTech companies who are insanely wealthy. Should Google reveal their search engine tech secrets? Can we apply this to Apple? Microsoft? Amazon? Meanwhile, the DNSBL industry has taken many hits lately because much of our pool of potential customers has massively shrunk in recent years - most business have migrated their email hosting to large providers such as O365, G-Suite, and to a lesser extend large companies like ProofPoint, Mimecast, etc - some of them are bringing in literally hundreds of millions, or even billions, in profits from email. I've lost count of how many subscribers invaluement has lost in the past few years because they moved their email to G-Suite or O365 and then no longer needed our services. So I'm not really in the mood to be told that I don't give back enough! (3) In the past few months in particular, it's been a struggle for invaluement precisely because so many spammers have started spamming MUCH MORE OFTEN via freemail accounts - ESPECIALLY GMAIL. So this means that they're essentially doing an "end run around" invaluement data, and that has harmed our business. It's directly causing fewer signups and even a dip in renewals (which are normally very strong). So to deal with that, I've spent the past few months doing a massive amount of "unbillable overhead" work - literally hundreds of hours, maybe into the thousands - working around the clock - developing an addition to invaluement that is basically a hashed email-address DNSBL (for authenticated email addresses, not forged ones). It's very close to an official release, and I'm very excited about the quality of this new list. I know invaluement isn't the first to create such a list, but I'm confident that this will be one of the best ones, if not the best one. Releasing this should be a big boost to the overall health of invaluement - but if this didn't get done - and we continued to lose relevancy/effectiveness due to so many spammers migrating to sending spam via freemail (especially gmail!) - much more often than they've done in previous years - invaluement would be run into the ground within MONTHS! (months, not years!) So maybe go pick on Google for not "sharing" more source code and not sharing more trade secrets? (This came an an inconvenient timing - that was basically chosen FOR us by Gmail's spam volumes! I was originally planning to expand our new SPBL data in other ways.) But, thankfully, with this new data that is near release, I think that invaluement will do well in the long run - but it's been a struggle in the meantime. (4) Is LinuxMagic software opensource? (not saying it's not, just curious now) -- Rob McEwen, invaluement ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 11/10/2021 10:35 PM, Chris via mailop wrote: On 2021-11-10 10:09 p.m., Collider via mailop wrote: Wait - Spamhaus dnsd is in C++? No. I'm the one who mentioned "C++", sorry if that wasn't accurate. It's been almost 4 years since I dug into the source code and made my own custom mods, so it's been a while. I'm 100% positive that I used the G++ program to compile it, so it was easy to get that mixed up since G++ is generally considered to be a C++ compiler, but it might be actually been C source code or something similar that I modified and that G++ compiled. So I'm now running an executable that was compiled with G++, fwiw. (even if the actual source code wasn't C++). I don't even remember what I ate for lunch this day last week, so remembering this kind of detail from 4 years ago isn't my strong suite! But I'll try to be more accurate in the future. -- Rob McEwen, invaluement ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-11-10 9:58 p.m., Michael Peddemors via mailop wrote: On 2021-11-10 11:47 a.m., Rob McEwen via mailop wrote: Already, the source/git are in too many multiple places. Whatever happened to giving back to opensource? I think you are building on other's 'hard-earned expertise' as well. Even despite how hard some people try to keep the opensource reasonably unified, things proliferate. As I understand it, MJT formally handed over leadership on rbldnsd *including* the associated mailing list to spamhaus. Yet, Debian appears to distribute a version of rbldnsd with a higher version number (0.999 on Ubuntu 20.04 - git maintainer our own Vsevolod) than spamhaus does (0.998b, with a 1.0 waiting in the wings). CentOS is .998b. The manual page of the debian distribution explicitly states it's for .998, so if there's any new features in .999, they're not documented. Can I scream now? ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-11-10 10:09 p.m., Collider via mailop wrote: Wait - Spamhaus dnsd is in C++? No. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
Wait - Spamhaus dnsd is in C++? On 11 November 2021 02:58:58 UTC, Michael Peddemors via mailop wrote: >On 2021-11-10 11:47 a.m., Rob McEwen via mailop wrote: >> The only issue here is that, for every user/customer that needs a unique >> key, an entirely different set of data has to be loaded into memory on >> the server. That's a huge limitation. It doesn't "scale". Therefore, for >> invaluement, in our new-ish direct query system (that started in 2018), >> that uses such unique keys for each DNS query customer, I had to >> basically custom program rbldnsd to overcome this. It took dozens of >> hours of very frustratingly-difficult programing, but that's partly >> because I'm not very good at C++! (by doing this myself, that was >> especially helpful for keeping this very hard-earned expertise in-house >> and very guarded!) > >Not sure why we/you should keep it 'inhouse', as it seems that all RBL >operators are adopting a similar strategy. > >We need more co-operative opensource collaboration, and ensure that >RBLDNSD doesn't end up with twenty forks, so it gets even better >performance, and simply has an option to treat all queries to a 'keyed' >subdomain, either to be served a single data source, or individual data >sources, based on a configuration item ;) > >And of course, the DSO support ;) > >Already, the source/git are in too many multiple places. Whatever >happened to giving back to opensource? I think you are building on >other's 'hard-earned expertise' as well. > > > > > > >-- >"Catch the Magic of Linux..." > >Michael Peddemors, President/CEO LinuxMagic Inc. >Visit us at http://www.linuxmagic.com @linuxmagic >A Wizard IT Company - For More Info http://www.wizard.ca >"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd. > >604-682-0300 Beautiful British Columbia, Canada > >This email and any electronic data contained are confidential and intended >solely for the use of the individual or entity to which they are addressed. >Please note that any views or opinions presented in this email are solely >those of the author and are not intended to represent those of the company. >___ >mailop mailing list >mailop@mailop.org >https://list.mailop.org/listinfo/mailop -- Sent from my Android device with K-9 Mail. Please excuse my brevity.___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-11-10 11:47 a.m., Rob McEwen via mailop wrote: The only issue here is that, for every user/customer that needs a unique key, an entirely different set of data has to be loaded into memory on the server. That's a huge limitation. It doesn't "scale". Therefore, for invaluement, in our new-ish direct query system (that started in 2018), that uses such unique keys for each DNS query customer, I had to basically custom program rbldnsd to overcome this. It took dozens of hours of very frustratingly-difficult programing, but that's partly because I'm not very good at C++! (by doing this myself, that was especially helpful for keeping this very hard-earned expertise in-house and very guarded!) Not sure why we/you should keep it 'inhouse', as it seems that all RBL operators are adopting a similar strategy. We need more co-operative opensource collaboration, and ensure that RBLDNSD doesn't end up with twenty forks, so it gets even better performance, and simply has an option to treat all queries to a 'keyed' subdomain, either to be served a single data source, or individual data sources, based on a configuration item ;) And of course, the DSO support ;) Already, the source/git are in too many multiple places. Whatever happened to giving back to opensource? I think you are building on other's 'hard-earned expertise' as well. -- "Catch the Magic of Linux..." Michael Peddemors, President/CEO LinuxMagic Inc. Visit us at http://www.linuxmagic.com @linuxmagic A Wizard IT Company - For More Info http://www.wizard.ca "LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd. 604-682-0300 Beautiful British Columbia, Canada This email and any electronic data contained are confidential and intended solely for the use of the individual or entity to which they are addressed. Please note that any views or opinions presented in this email are solely those of the author and are not intended to represent those of the company. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 11/10/2021 2:31 PM, Chris via mailop wrote: like any other DNS server, you *can* implement a single key by putting your DNSBL zone under a name like ".example.com" The only issue here is that, for every user/customer that needs a unique key, an entirely different set of data has to be loaded into memory on the server. That's a huge limitation. It doesn't "scale". Therefore, for invaluement, in our new-ish direct query system (that started in 2018), that uses such unique keys for each DNS query customer, I had to basically custom program rbldnsd to overcome this. It took dozens of hours of very frustratingly-difficult programing, but that's partly because I'm not very good at C++! (by doing this myself, that was especially helpful for keeping this very hard-earned expertise in-house and very guarded!) I'm not sure how the Rspamd version handles that memory issue? But for what I do, there's just one dataset in memory for each zone that serves queries that have _different_ license keys in the license key DNS-label part (but where the zone name is /otherwise/ the same). So this is very efficient! -- Rob McEwen, invaluement ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
I've checked. The Spamhaus publicly distributed version of rbldnsd does not support multiple DQS-style keys. But, like any other DNS server, you *can* implement a single key by putting your DNSBL zone under a name like ".example.com", and as long as the mail servers that are supposed to *know* that key don't publish them by accident in error messages it's reasonably secure and nobody else can query it even if they know the base name. On 2021-11-04 4:08 p.m., Vsevolod Stakhov via mailop wrote: On 04/11/2021 19:53, John Levine wrote: It appears that Vsevolod Stakhov via mailop said: Rspamd RBLDNSD version [1] can do it as well. It is also much faster than the original one (like 4 times faster) and allows dynamic updates. And unlike Spamhaus stuff, it is free and open source, of course. Um ... https://github.com/spamhaus/rbldnsd Do you see any of the features discussed in this thread in the Spamhaus rbldnsd repo? And we all assume that those features are 'just' not published, don't we? On the contrary, my version supports access limitation defined by 'DNS keys' by aclkey zone extension. I have done quite a lot of other improvements there, and have some more ideas about it (like rate-limiting and multiprocess mode). ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 04/11/2021 19:53, John Levine wrote: > It appears that Vsevolod Stakhov via mailop said: >> Rspamd RBLDNSD version [1] can do it as well. It is also much faster >> than the original one (like 4 times faster) and allows dynamic updates. >> And unlike Spamhaus stuff, it is free and open source, of course. > > Um ... https://github.com/spamhaus/rbldnsd > Do you see any of the features discussed in this thread in the Spamhaus rbldnsd repo? And we all assume that those features are 'just' not published, don't we? On the contrary, my version supports access limitation defined by 'DNS keys' by aclkey zone extension. I have done quite a lot of other improvements there, and have some more ideas about it (like rate-limiting and multiprocess mode). ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
It appears that Vsevolod Stakhov via mailop said: >Rspamd RBLDNSD version [1] can do it as well. It is also much faster >than the original one (like 4 times faster) and allows dynamic updates. >And unlike Spamhaus stuff, it is free and open source, of course. Um ... https://github.com/spamhaus/rbldnsd R's, John ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
It appears that Michael Peddemors via mailop said: > [ put a per-customer token in the DNSBL name ] >I do believe so ;) It isn't a perfect secret key of course, more of an >obfuscated key, but it is an easy way to discourage abuse, and allows >queries to be allowed via shared or open resolvers in a way that a >single end point query does not. It doesn't prevent people from sharing keys but it makes it fairly easy to detect, which for this purpose is good enough. >While not perfect, it is a smart technique. The only thing is the >actual DNS server itself has to either be modified to parse and check >the key, or it has to set up a 'zone' dynamically as people register for >their keys. I believe Spamhaus uses a customized version of rbldnsd so the whole idea of zones doesn't apply. It parses the queries and synthesizes the responses on the fly. It's not a perfect implementation of the DNS protocol but it's close enough. R's, John ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 04/11/2021 14:28, Bill Cole via mailop wrote: > On 2021-11-04 at 10:07:04 UTC-0400 (Thu, 4 Nov 2021 09:07:04 -0500) > Larry M. Smith via mailop > is rumored to have said: > >> Isn't this how Spamhaus runs their DQS service? > > Yes. It is how they've run non-free access for well over a decade. A > solved problem with broad deployment. > > How they handle the backend is unclear. Their scale -- thousands of > unique users each querying a collection of large zones that are each > replicated for each user -- presents a special challenge that would be > daunting to meet with common DNS server software. Whatever they've set > up seems to work well. > Rspamd RBLDNSD version [1] can do it as well. It is also much faster than the original one (like 4 times faster) and allows dynamic updates. And unlike Spamhaus stuff, it is free and open source, of course. [1]: https://github.com/rspamd/rbldnsd ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 11/4/21 3:21 PM, Michael Peddemors via mailop wrote: On 2021-11-04 7:07 a.m., Larry M. Smith via mailop wrote: While not perfect, it is a smart technique. The only thing is the actual DNS server itself has to either be modified to parse and check the key, or it has to set up a 'zone' dynamically as people register for their keys. If the later of course, you could have a very large set of zones to be maintained, if you are popular enough, one for each registered user. In the same way, the smtp server should not give the list with the key as an answer to the potential spammer. Like "550 blacklisted at hereismykeyfeelfreetouseit.zen.dq.spamhaus.net". ;) smime.p7s Description: S/MIME Cryptographic Signature ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-11-04 at 10:07:04 UTC-0400 (Thu, 4 Nov 2021 09:07:04 -0500) Larry M. Smith via mailop is rumored to have said: Isn't this how Spamhaus runs their DQS service? Yes. It is how they've run non-free access for well over a decade. A solved problem with broad deployment. How they handle the backend is unclear. Their scale -- thousands of unique users each querying a collection of large zones that are each replicated for each user -- presents a special challenge that would be daunting to meet with common DNS server software. Whatever they've set up seems to work well. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
The spamhaus supported version of rbldnsd may understand the use of keys in this fashion. If an ordinary DNS server is configured correctly, it should return NXDOMAIN for those who don't know the key (the DNS server is SOA'd to the base name, not the base+key. But this will be a problem if you want to have multiple keys. On 2021-11-04 9:28 a.m., Nicolas JEAN via mailop wrote: On 03/11/2021 15:20, Bill Cole via mailop wrote: On 2021-11-03 at 05:42:36 UTC-0400 (Wed, 3 Nov 2021 10:42:36 +0100) Nicolas JEAN via mailop is rumored to have said: On 15/10/2021 23:22, Paul Gregg via mailop wrote: The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. Nice trick. :) Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Not software modification, just normal configuration. In Postfix, postscreen_dnsbl_sites, reject_rbl_client, and every other directive to do DNSBL queries takes arbitrary zone labels as the basis of queries so you can just use secretclientkey.dnsbl.example.com instead of dnsbl.example.com. Postfix also has the ability to customize the error message sent to listed clients so that you do not reveal your client key. Thanks Paul and Bill, you're right. So it just amounts to some DNS config, and writing the DNS server software that recognizes and acts upon the given 'secret', then. Regards, Nico ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-11-04 7:07 a.m., Larry M. Smith via mailop wrote: On 11/3/2021, Nicolas JEAN via mailop wrote: On 15/10/2021 23:22, Paul Gregg via mailop wrote: (snip) Sorry for the late reply. The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. Nice trick. :) Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Still an idea to keep in mind. Because of how DNS works, the source IP address isn't available anyway in a usual, unmodified postfix DNS query. Isn't this how Spamhaus runs their DQS service? I do believe so ;) It isn't a perfect secret key of course, more of an obfuscated key, but it is an easy way to discourage abuse, and allows queries to be allowed via shared or open resolvers in a way that a single end point query does not. While not perfect, it is a smart technique. The only thing is the actual DNS server itself has to either be modified to parse and check the key, or it has to set up a 'zone' dynamically as people register for their keys. If the later of course, you could have a very large set of zones to be maintained, if you are popular enough, one for each registered user. -- "Catch the Magic of Linux..." Michael Peddemors, President/CEO LinuxMagic Inc. Visit us at http://www.linuxmagic.com @linuxmagic A Wizard IT Company - For More Info http://www.wizard.ca "LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd. 604-682-0300 Beautiful British Columbia, Canada This email and any electronic data contained are confidential and intended solely for the use of the individual or entity to which they are addressed. Please note that any views or opinions presented in this email are solely those of the author and are not intended to represent those of the company. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-11-04 at 09:28:28 UTC-0400 (Thu, 4 Nov 2021 14:28:28 +0100) Nicolas JEAN via mailop is rumored to have said: On 03/11/2021 15:20, Bill Cole via mailop wrote: On 2021-11-03 at 05:42:36 UTC-0400 (Wed, 3 Nov 2021 10:42:36 +0100) Nicolas JEAN via mailop is rumored to have said: On 15/10/2021 23:22, Paul Gregg via mailop wrote: The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. Nice trick. :) Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Not software modification, just normal configuration. In Postfix, postscreen_dnsbl_sites, reject_rbl_client, and every other directive to do DNSBL queries takes arbitrary zone labels as the basis of queries so you can just use secretclientkey.dnsbl.example.com instead of dnsbl.example.com. Postfix also has the ability to customize the error message sent to listed clients so that you do not reveal your client key. Thanks Paul and Bill, you're right. So it just amounts to some DNS config, and writing the DNS server software that recognizes and acts upon the given 'secret', then. There's no need for anything special in DNS server software unless you're looking to scale to hundreds of unique users. Any DNS server capable of serving hundreds of zones can serve hundreds of identical zones with different names. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 11/3/2021, Nicolas JEAN via mailop wrote: On 15/10/2021 23:22, Paul Gregg via mailop wrote: (snip) Sorry for the late reply. The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. Nice trick. :) Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Still an idea to keep in mind. Because of how DNS works, the source IP address isn't available anyway in a usual, unmodified postfix DNS query. Isn't this how Spamhaus runs their DQS service? ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 03/11/2021 15:20, Bill Cole via mailop wrote: On 2021-11-03 at 05:42:36 UTC-0400 (Wed, 3 Nov 2021 10:42:36 +0100) Nicolas JEAN via mailop is rumored to have said: On 15/10/2021 23:22, Paul Gregg via mailop wrote: The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. Nice trick. :) Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Not software modification, just normal configuration. In Postfix, postscreen_dnsbl_sites, reject_rbl_client, and every other directive to do DNSBL queries takes arbitrary zone labels as the basis of queries so you can just use secretclientkey.dnsbl.example.com instead of dnsbl.example.com. Postfix also has the ability to customize the error message sent to listed clients so that you do not reveal your client key. Thanks Paul and Bill, you're right. So it just amounts to some DNS config, and writing the DNS server software that recognizes and acts upon the given 'secret', then. Regards, Nico OpenPGP_0x23459069119D37B6.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-11-03 at 05:42:36 UTC-0400 (Wed, 3 Nov 2021 10:42:36 +0100) Nicolas JEAN via mailop is rumored to have said: On 15/10/2021 23:22, Paul Gregg via mailop wrote: On 10/4/21 18:52, Leandro Santiago via mailop wrote: Hi list, How feasible to you folks think having a DNSBL server that accepts only connections from a group of IP is? By that I mean that the server will accept (UDP) DNS requests from an "allow list", refusing requests from anyone else (basically answering "nothing" from any dns question from other IP addresses). I am using the IP from the UDP request packet to perform the "authentication". This is for a DNSBL which is not supposed to be public, although the DNS server is accessible publicly on the internet. I want to keep the DNSBL "spec", so for a request: A 44.33.22.11.myserver.example.com. I'll answer, in case 11.22.33.44 is "blocklisted": A 127.0.0.2 Sorry for the late reply. The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. Nice trick. :) Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Not software modification, just normal configuration. In Postfix, postscreen_dnsbl_sites, reject_rbl_client, and every other directive to do DNSBL queries takes arbitrary zone labels as the basis of queries so you can just use secretclientkey.dnsbl.example.com instead of dnsbl.example.com. Postfix also has the ability to customize the error message sent to listed clients so that you do not reveal your client key. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 03/11/2021 09:42, Nicolas JEAN via mailop wrote: Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Still an idea to keep in mind. Because of how DNS works, the source IP address isn't available anyway in a usual, unmodified postfix DNS query. Why would it need modifications? It's for a DNSBL, not general DNS queries, so you have to tell Postfix the DNSBL to use anyway. So, rather than saying 'check the DNSBL called myspamblock.com', you 'check the DNSBL called hegr27dm31.myspamblock.com' (where 'hegr27dm31' is the 'secret key') -- Paul Paul Smith Computer Services supp...@pscs.co.uk - 01484 855800 -- Paul Smith Computer Services Tel: 01484 855800 Vat No: GB 685 6987 53 Sign up for news & updates at http://www.pscs.co.uk/go/subscribe ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 15/10/2021 23:22, Paul Gregg via mailop wrote: On 10/4/21 18:52, Leandro Santiago via mailop wrote: Hi list, How feasible to you folks think having a DNSBL server that accepts only connections from a group of IP is? By that I mean that the server will accept (UDP) DNS requests from an "allow list", refusing requests from anyone else (basically answering "nothing" from any dns question from other IP addresses). I am using the IP from the UDP request packet to perform the "authentication". This is for a DNSBL which is not supposed to be public, although the DNS server is accessible publicly on the internet. I want to keep the DNSBL "spec", so for a request: A 44.33.22.11.myserver.example.com. I'll answer, in case 11.22.33.44 is "blocklisted": A 127.0.0.2 Sorry for the late reply. The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. Nice trick. :) Unfortunately, it seems that it would require modifications to e.g. postfix, or other software, in order to add that identifying string to the DNS query. Still an idea to keep in mind. Because of how DNS works, the source IP address isn't available anyway in a usual, unmodified postfix DNS query. Cheers, Nico OpenPGP_0x23459069119D37B6.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On Wed, Oct 06, 2021 at 10:22:11AM +0200, Leandro Santiago via mailop wrote: > Thank you all who shared their knowledge. > > We decided, on our solution, to go away from the DNSBL approach which has > way too many caveats and are now experimenting with solutions on a higher > level on the network stack. > > On 10/4/21 18:52, Leandro Santiago via mailop wrote: > > Hi list, > > > > How feasible to you folks think having a DNSBL server that accepts only > > connections from a group of IP is? > > > > By that I mean that the server will accept (UDP) DNS requests from an > > "allow list", refusing requests from anyone else (basically answering > > "nothing" from any dns question from other IP addresses). I am using the > > IP from the UDP request packet to perform the "authentication". > > > > This is for a DNSBL which is not supposed to be public, although the DNS > > server is accessible publicly on the internet. I want to keep the DNSBL > > "spec", so for a request: > > > > A 44.33.22.11.myserver.example.com. > > > > I'll answer, in case 11.22.33.44 is "blocklisted": > > > > A 127.0.0.2 Sorry for the late reply. The trick to this is not to limit by IP address - but to implement service (API) keys. e.g. each authorised user is given a key e.g. sj3Fa3Gomd937Z12 Then they make queries for 44.33.22.11.sj3Fa3Gomd937Z12.myserver.example.com. That way you don't care what IP it comes from, but you know who it is. PG Sent via Proofpoint Essentials https://proofpointessentials.co.uk ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
Good idea. RBLs are brilliant but when Vixie invented the idea and the Apache SpamAssassin team implemented it some 2 decades ago, I wrote something to the effect of what a (brilliant) hack. We need to replace this with something in a year. Now, many many years later and several DDOSes against RBL operators like STORM network, the use of DNS for RBLs remains :-) Regards, KAM On 10/6/2021 4:22 AM, Leandro Santiago via mailop wrote: Thank you all who shared their knowledge. We decided, on our solution, to go away from the DNSBL approach which has way too many caveats and are now experimenting with solutions on a higher level on the network stack. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
Thank you all who shared their knowledge. We decided, on our solution, to go away from the DNSBL approach which has way too many caveats and are now experimenting with solutions on a higher level on the network stack. On 10/4/21 18:52, Leandro Santiago via mailop wrote: Hi list, How feasible to you folks think having a DNSBL server that accepts only connections from a group of IP is? By that I mean that the server will accept (UDP) DNS requests from an "allow list", refusing requests from anyone else (basically answering "nothing" from any dns question from other IP addresses). I am using the IP from the UDP request packet to perform the "authentication". This is for a DNSBL which is not supposed to be public, although the DNS server is accessible publicly on the internet. I want to keep the DNSBL "spec", so for a request: A 44.33.22.11.myserver.example.com. I'll answer, in case 11.22.33.44 is "blocklisted": A 127.0.0.2 Or no answer at all in case 11.22.33.44 is not listed, or the request was refused. The issue I believe I'll find (if I understood DNS properly) is that when my client makes the request, it'll hop through several DNS servers (first, operating system's cache, a router, ISP and so on...) so that the the final request to my DNS server will be done by some DNS server in the middle, meaning my initial IP address will be lost in the way. Would it be possible to make it work without telling my client to use `example.com` as nameserver? I don't mind if my client caches requests, as I plan to use reasonable TTL to make sure the requests are valid. I was thinking on adding my DNS server IP as a `NS` record for `example.com.`, but I confess I have not understood the DNS protocol well enough to try it. I am limited to using DNS over UDP (instead of fancy things such as DNS over HTTPS) as my clients are mostly Postfix instances, and so far it supports only plain UDP DNS lookup. I suspect the whole idea is just not feasible, but I wantd some second opinion from someone who has more knowledge about it or who has implemented something similar. PS: I am not using bind or any other off-the-shelf solution, but instead a custom DNS server implementation powered by go-dns, the same library used by PowerDNS. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop -- Regards, Leandro Santiago Software Craftsman at Lightmeter https://lightmeter.io OpenPGP_0xAB5F702209190A96.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
On 2021-10-04 at 12:52:40 UTC-0400 (Mon, 4 Oct 2021 18:52:40 +0200) Leandro Santiago via mailop is rumored to have said: > Hi list, > > How feasible to you folks think having a DNSBL server that accepts only > connections from a group of IP is? Works for me. I've been running a local DNSBL for almost 20y. The only problem is that the name of the DNSBL has leaked to public sources, resulting in a bunch of idiots trying to query it, including (at times) idiots who run "Multi-Blacklist Checkers" incompetently. Nothing I do on the nameserver fixes that, including refusing queries and answering them in ways designed to cause a nuisance for the querying parties. So, I do not recommend citing the DNSBL zone anywhere, even if it is only in MTA rejection messages. [how DNSBLs work snipped...] > The issue I believe I'll find (if I understood DNS properly) is that when my > client makes the request, it'll hop through several DNS servers (first, > operating system's cache, a router, ISP and so on...) so that the the final > request to my DNS server will be done by some DNS server in the middle, > meaning my initial IP address will be lost in the way. Yes. If clients do not run their own recursing resolvers, this will happen. So they should run their own local recursing resolver. Every mail server should use one anyway, as DNS performance and a nearby DNS cache are helpful and you cannot use shared public resolvers if you want to use URIBL or Spamhaus. > Would it be possible to make it work without telling my client to use > `example.com` as nameserver? > > I don't mind if my client caches requests, as I plan to use reasonable TTL to > make sure the requests are valid. > > I was thinking on adding my DNS server IP as a `NS` record for > `example.com.`, but I confess I have not understood the DNS protocol well > enough to try it. Only useful if your server is in fact authoritative for the whole domain. Does not solve the core issue: anyone not using their own recursive resolver will generate queries to your server from whatever recursive resolver they ultimately use. > I am limited to using DNS over UDP (instead of fancy things such as DNS over > HTTPS) as my clients are mostly Postfix instances, and so far it supports > only plain UDP DNS lookup. Postfix uses anything the OS resolver supports. If your OS resolver can do DNS over HTTPS, so can Postfix. However, you SHOULD NOT use DNS over HTTPS for Postfix or any other MTA. > I suspect the whole idea is just not feasible, but I wantd some second > opinion from someone who has more knowledge about it or who has implemented > something similar. It's entirely feasible. What is NOT feasible is running a performant MTA without a local recursing and caching DNS resolver. > PS: I am not using bind or any other off-the-shelf solution, but instead a > custom DNS server implementation powered by go-dns, the same library used by > PowerDNS. I use BIND for my do-it-all nameserver, but I can't recommend it as just a way to add your own recursing caching resolver. For that generally I recommend Unbound, which is designed for such tasks. It is my understanding that PowerDNS also offers a caching recursive resolver, so you could use that. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
Am 04.10.21 um 18:52 schrieb Leandro Santiago via mailop: > Hi list, > > How feasible to you folks think having a DNSBL server that accepts only > connections from a group of IP is? I'm afraid it most likely won't work. Even if you use your server as the NS record for the DNSBL domain, clients will go through their configured DNS servers and let them access your server. Normally, this has huge caching advantages, you only want to walk the chain of DNS servers and glue records when debugging DNS issues. If the clients all have very standard Linux configurations and are willing to tweak their BIND configuration, they may follow the recipe at https://serverfault.com/questions/18748/overriding-some-dns-entries-in-bind-for-internal-networks. However, I wouldn't consider this if they are faint of heart :-) Cheers, Hans-Martin ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Feasibility of a private DNSBL
This is a standard feature of RBLDNSD, we use it all the time. You can use the ACL either to refuse, ignore, or accept from IP Ranges. But of course, re-inventing the wheel, when there are so many good RBL's out there (including our own ;) not sure what your differentiator would be. You can use your own RBLDNSD server for multiple lists of course, and use the return codes to determine how you handle the issue of listing. Take a look at the documentation for RBLDNSD, it isn't hard to set up.. On 2021-10-04 9:52 a.m., Leandro Santiago via mailop wrote: Hi list, How feasible to you folks think having a DNSBL server that accepts only connections from a group of IP is? By that I mean that the server will accept (UDP) DNS requests from an "allow list", refusing requests from anyone else (basically answering "nothing" from any dns question from other IP addresses). I am using the IP from the UDP request packet to perform the "authentication". This is for a DNSBL which is not supposed to be public, although the DNS server is accessible publicly on the internet. I want to keep the DNSBL "spec", so for a request: A 44.33.22.11.myserver.example.com. I'll answer, in case 11.22.33.44 is "blocklisted": A 127.0.0.2 Or no answer at all in case 11.22.33.44 is not listed, or the request was refused. The issue I believe I'll find (if I understood DNS properly) is that when my client makes the request, it'll hop through several DNS servers (first, operating system's cache, a router, ISP and so on...) so that the the final request to my DNS server will be done by some DNS server in the middle, meaning my initial IP address will be lost in the way. Would it be possible to make it work without telling my client to use `example.com` as nameserver? I don't mind if my client caches requests, as I plan to use reasonable TTL to make sure the requests are valid. I was thinking on adding my DNS server IP as a `NS` record for `example.com.`, but I confess I have not understood the DNS protocol well enough to try it. I am limited to using DNS over UDP (instead of fancy things such as DNS over HTTPS) as my clients are mostly Postfix instances, and so far it supports only plain UDP DNS lookup. I suspect the whole idea is just not feasible, but I wantd some second opinion from someone who has more knowledge about it or who has implemented something similar. PS: I am not using bind or any other off-the-shelf solution, but instead a custom DNS server implementation powered by go-dns, the same library used by PowerDNS. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop -- "Catch the Magic of Linux..." Michael Peddemors, President/CEO LinuxMagic Inc. Visit us at http://www.linuxmagic.com @linuxmagic A Wizard IT Company - For More Info http://www.wizard.ca "LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd. 604-682-0300 Beautiful British Columbia, Canada This email and any electronic data contained are confidential and intended solely for the use of the individual or entity to which they are addressed. Please note that any views or opinions presented in this email are solely those of the author and are not intended to represent those of the company. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
