Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 11:14 AM, David F. Skoll wrote: On Tue, 04 Jan 2011 11:01:52 -0500 Rob McEwen r...@invaluement.com wrote I've thought this through and... best case scenario is that spammers then get 5+ years of play time because it will take at least that time for those other techniques to catch up. Umm.. no. We have plenty of effective techniques we're using right now that don't rely on DNSBLs. I think if we stopped using DNSBLs completely, a bit more spam would leak through, but it wouldn't be catastrophic. Yes, it would be catastrophic. For one, it would bring the large ISPs down to their knees. Easily! Heck, currently, I know of one extremely famous and large ISP who isn't even willing to parse out the domains in incoming messages to check against locally hosted URI blacklists because that would mean too much resources per message. (and that process is extremely fast and efficient!). Many smaller ISPs *depend* on blocking at connection time with Zen and would likewise crash burn if DNSBL-blocking at connection time wasn't feasible. When we are left with only whitelists and no blacklists, an interesting problem will happen... there will be extreme prejudice against ALL new IPs not already whitelisted. Life will become more difficult, but it's not all doom-and-gloom. By default, you should be able to get on the whitelist just by asking. If it turns out you've abused the trust, you get banned for a long time. That's essentially how the Spamhaus Whitelist works. You're exactly right. The reliance on whitelists would grow... those not on whitelists (like small businesses and start-ups) would be screwed. This would lead to a chicken/egg problems... how do you build up good reputation to get on a whitelist if you can't sent mail until you're on one. That will lead to a need for a more easy on whitelisting process.. abet even if for a less trusted level. But spammers will then often sneak onto that 1st tier... but at least it is better than dealing with the massive numbers of IPs no on there at all! But do you see where this is leading? right back to my original idea. Looks like you agree we'll get to my original idea anyways, one way or another. (although, I think it would be a lot less messy... and spammers would get much less listwashing accomplished... if we took a more direct route!) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Tue, Jan 4, 2011 at 8:27 AM, Jason Haar jason.h...@trimble.co.nz wrote: This is a great topic! Is this been discussed at the IETF level? This is much bigger than SA. From the sounds of this thread, spam under ipv6 is going to be almost an *infinitely* bigger problem than ipv4. What about The IETF is where it's coming from (the ASRG mailing list, to be more precise). I just pointed to the discussion here on the SA list because SA will most likely be a prime user of whatever new protocols / designs being RFCed. As far as I could follow the discussion, I have the feeling that there is 1) some common understanding of the issues (sheer size of the IP space, which makes it hard to manage data in regular operations, and even more so when confronted with intentional or unintentional abusive behavior), and 2) some common understanding of the solution space (from make everything new via pimp up the protocol to do not change anything and hope for the best, with most arguments centering around the middle ground). Thanks for all your arguments. -- Matthias
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Mon, Jan 3, 2011 at 9:27 PM, Jason Haar jason.h...@trimble.co.nz wrote: On 01/04/2011 04:50 PM, Dave Pooser wrote: Frankly, I'd think that besides costing the spammers money (a good thing in and of itself) ...spammers steal other people's resources - so they'll pay nothing... The best case scenario we can ever hope for is that they will be stuck sending all their spam using the From: address and SMTP server of the infected host - nothing better is possible, unless you can figure out how to stop 100% of humanity clicking on %*# executables. Some ISP's appear to be doing a much better job at preventing spam-through-official-SMTP-servers than they used to. I just now noticed that rr.com appears to be using Cloudmark on customer mail leaving their official MTA's. Looking through my logs, it appears very little of my spam is coming from official rr.com MTA's these days. This is a good sign. Now why can't Yahoo do this!? =) Warren
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 1:57 AM, John Levine wrote: I also don't think it's very realistic to expect that there will be a master mail host file distributed periodically like HOSTS.TXT was. There's a reason that the DNS was invented, and at the time it was, there were a whole lot less hosts on the net than there are mail hosts now. Others will take this and publish it via DNS (even if just for their own customers), which is a good thing. Also, such a list could be a raw list of just the IPs, nothing else, fwiw. (Obviously, every single spam filter instance worldwide wouldn't try to download that entire list.) Finally, I presume everyone is aware that there is no possibility whatsoever that RIRs will get into the business of selling mail host IPs, so this entire subthread is 100% hypothetical. If so, then a (less advantageous) private solution will hopefully develop. But talking about realistic, the status quo is already not realistic, even with the good ideas that you proposed, which did improve on this problems in _some_ aspects. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
A couple more cents on this topic... If the problem is blowing DNS caches, then one solution is to query only authoritative name servers. Spamhaus, for example, permits 300,000 free queries per day. I bet many small sites will be under this limit even if they query Spamhaus directly with no caching. For larger sites, you rsync to your own authoritative server. Again, no caching. This will put a larger load on DNS[BW]Ls, but I think it's manageable. After all, the total volume of DNS[BW]L queries from mail servers even without caching is probably very much less than the total volume of queries that go to the root name servers and they seem to cope. (I don't mean to discount the massive job of running the root name servers. It just means that people running DNS[BW]Ls will have to be prepared to make a significant infrastructure investment. It also means that fewer people will be able to set up lists on a whim and there might be fewer free lists. Those might be Good Things if list quality improves.) Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Tue, 4 Jan 2011, David F. Skoll wrote: If the problem is blowing DNS caches, then one solution is to query only authoritative name servers. After all, the total volume of DNS[BW]L queries from mail servers even without caching is probably very much less than the total volume of queries that go to the root name servers and they seem to cope. You can't compare them. The nature of the queries is vastly different - the root nameservers only get queries like where are the authoritative DNS servers for impsec.org? Only querying the authoritative RBL server discards the distributed caching feature of DNS, which is a primary benefit of using DNS. It will greatly increase the load on those authoritative servers, likely leading those who provide them to alter their free query policies in order to recover their suddenly-increased costs of operation. DNS needs to deal with an exponentially-increased address space regardless of how RBLs behave. Perhaphs DNS caching needs to be partitioned so that a huge number of queries on *.spamhaus.org don't blow everything else out of the cache. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Vista: because the audio experience is *far* more important than network throughput. --- 13 days until Benjamin Franklin's 305th Birthday
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Tue, 4 Jan 2011 06:18:55 -0800 (PST) John Hardin jhar...@impsec.org wrote: DNS needs to deal with an exponentially-increased address space regardless of how RBLs behave. Perhaphs DNS caching needs to be partitioned so that a huge number of queries on *.spamhaus.org don't blow everything else out of the cache. Right, but once your cache is blown, you're back to always querying the authoritative server. John Levine proposes a fix with a clever way to represent many entries with a small number of queries so you don't blow your cache. I think making zone files available for download so you can run your own authoritative servers is another good approach, especially for whitelists. Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 9:31 AM, David F. Skoll wrote: Right, but once your cache is blown, you're back to always querying the authoritative server. John Levine proposes a fix with a clever way to represent many entries with a small number of queries so you don't blow your cache. I think making zone files available for download so you can run your own authoritative servers is another good approach, especially for whitelists. What I'm about to say could have been a reply to ANY of the past few posts... if the volume of unique IPv6 sending IPs causes either (a) DNS caches to get blown, or (b) requires John Levine's solution to prevent that... then... game over.. the spammers have already won. And they are quite amused right now reading us discuss all different ways to rearrange the deck chairs on the Titanic. For example, if this happens, then... (1) effective DNSBLs will likewise bloat to such a large size that the resource requirements for running them (and transferring their data) will be insane (2) David mentioned zone file transfers... but that zone file would likewise be massively large and any client trying to load it would also have to consume large resources trying to load it. (this would also cause problems trying to sync DNS mirrors!). Consider also those IPs which ought to be blacklisted quickly, but don't because of having to wait on the previous mirror update? (3) What do we do about spammers who send spams to 100K or even a million addresses, using a unique IP for each recipient... and then list wash off the addresses which match up with those IPs that get blacklisted? Sure, you could blacklist whole blocks of that spammer's IPs.. but then you have to be able to do that without causing collateral damage in other situations where spammers and legit sender share IPs. Without going into details, tactics exist to somewhat deal with this... but massive numbers of e-mail addresses would get listwashed at the beginning stages of each campaign. IOW, no matter what, large-scale and EASY listwashing would be the norm with IPv6. (4) No matter how good you deal with the previous idea, you're STILL stuck with massive numbers of never-to-be-seen-again IPs bloating IPv6 blacklists! (again, the spammers are laughing at us!) (5) If anyone thought that my proposed solution a few posts back was unrealistic... consider that the extremely inadequate partial fix solutions that others are proposing involve even MORE intrusive changes to the standards/behaviors of our various systems (filters, blacklists, dns servers, dns protocol, mail servers, etc) As I said earlier, ALL of these problems have the root cause of too large a potential pool of sending IPs. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Tue, 04 Jan 2011 10:34:43 -0500 Rob McEwen r...@invaluement.com wrote: game over.. the spammers have already won. And they are quite amused right now reading us discuss all different ways to rearrange the deck chairs on the Titanic. We are talking at cross-purposes here, but I think we mostly agree. :) (2) David mentioned zone file transfers... but that zone file would likewise be massively large. No... I mentioned that zone file transfers are useful mainly for whitelists, which are likely to be of manageable size and not likely to be updated very frequently. I agree that it's probably eventually game over for DNSBLs, but not for DNSWLs. DNSBLs are a pretty effective first-line defense against spam, but they will gradually become less and less effective as IPv6 becomes more heavily adopted. That just means we'll have to emphasize different techniques. It doesn't mean doom. Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Tue, 4 Jan 2011, David F. Skoll wrote: On Tue, 4 Jan 2011 06:18:55 -0800 (PST) John Hardin jhar...@impsec.org wrote: DNS needs to deal with an exponentially-increased address space regardless of how RBLs behave. Perhaphs DNS caching needs to be partitioned so that a huge number of queries on *.spamhaus.org don't blow everything else out of the cache. Right, but once your cache is blown, you're back to always querying the authoritative server. John Levine proposes a fix with a clever way to represent many entries with a small number of queries so you don't blow your cache. In the vein of DNS changes needed for IPv6 (vs. simply SA and DNSBLs) what _other_ applications would benefit from JL's tree proposal? (I confess I haven't read the paper yet...) I think making zone files available for download so you can run your own authoritative servers is another good approach, especially for whitelists. Oh, agreed. But I don't think it's the _only_ alternative. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Health Care _is_ a right - the government has no business keeping you from getting it. But forcing somebody else to pay for your health care at gunpoint (i.e. through taxation) is _not_ a right. --- 13 days until Benjamin Franklin's 305th Birthday
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/4/2011 10:43 AM, David F. Skoll wrote: I agree that it's probably eventually game over for DNSBLs, but not for DNSWLs. DNSBLs are a pretty effective first-line defense against spam, but they will gradually become less and less effective as IPv6 becomes more heavily adopted. That just means we'll have to emphasize different techniques. It doesn't mean doom. I've thought this through and... best case scenario is that spammers then get 5+ years of play time because it will take at least that time for those other techniques to catch up. Great damage will happen in the meantime. In the short term, old habits die hard. Without something like the solution I proposed, blacklists will die a slow and painful death. And most of the negatives I mentioned happening will STILL occur during the transition. When we are left with only whitelists and no blacklists, an interesting problem will happen... there will be extreme prejudice against ALL new IPs not already whitelisted. This will create a chicken/egg problem whereby a new startup company with new IPs will be screwed. (plus, small businesses will also have a hard time getting respect as they have have difficulty getting enough traction to get their IPs whitelisted!) Then, to fix the chicken/egg problem, someone will come along and create some kind of IPv6 senders list for new IPs that haven't been whitelisted yet. It will be extremely similar to my original proposed solution, since spammers will have no choice but to try to get on the IPv6 senders list list, too--but will be extremely limited in the volume of IPs they get on that that list. We'll then come full circle... except going with (something like) my proposed solution *now* instead of *then* means that we wouldn't have to have destroyed blacklists (and set spam filtering years back) in the meantime. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Tue, 04 Jan 2011 11:01:52 -0500 Rob McEwen r...@invaluement.com wrote: I've thought this through and... best case scenario is that spammers then get 5+ years of play time because it will take at least that time for those other techniques to catch up. Umm.. no. We have plenty of effective techniques we're using right now that don't rely on DNSBLs. I think if we stopped using DNSBLs completely, a bit more spam would leak through, but it wouldn't be catastrophic. When we are left with only whitelists and no blacklists, an interesting problem will happen... there will be extreme prejudice against ALL new IPs not already whitelisted. Life will become more difficult, but it's not all doom-and-gloom. By default, you should be able to get on the whitelist just by asking. If it turns out you've abused the trust, you get banned for a long time. That's essentially how the Spamhaus Whitelist works. Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Le 04/01/2011 17:01, Rob McEwen a écrit : I've thought this through and... best case scenario is that spammers then get 5+ years of play time because it will take at least that time for those other techniques to catch up. Great damage will happen in the meantime. That scenario assumes rapid adoption of IPv6 by consumer-oriented ISPs (since most spam is sent via botnets on consumer PCs), and I'm not at all convinced we'll see that happening fast despite the imminent exhaustion of IPv4 address space. Instead, IMO, IPv6 will be deployed to end users only very gradually, giving the anti-spam community plenty of time to keep pace with new spamming techniques. John. -- -- Over 4000 webcams from ski resorts around the world - www.snoweye.com -- Translate your technical documents and web pages- www.tradoc.fr
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Tue, 04 Jan 2011 11:01:52 -0500 Rob McEwen r...@invaluement.com wrote: When we are left with only whitelists and no blacklists, an interesting problem will happen... there will be extreme prejudice against ALL new IPs not already whitelisted. This will create a chicken/egg problem whereby a new startup company with new IPs will be screwed. (plus, small businesses will also have a hard time getting respect as they have have difficulty getting enough traction to get their IPs whitelisted!) It could be done as Whitelisted-IP || (Full-Circle-DNS ! Blacklisted-Domain)
DNS cache efficiency for low-TTL records (was Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01)
On Tue, 4 Jan 2011 06:18:55 -0800 (PST)
John Hardin jhar...@impsec.org wrote:
[DFS says all queries should be to authoritative name servers to avoid
cache blowouts.]
You can't compare them. The nature of the queries is vastly different
- the root nameservers only get queries like where are the
authoritative DNS servers for impsec.org?
Well. I ran a little experiment. I took a day's worth of logs from our
small non-busy mail server and looked at all the incoming mail. Spamhaus
appears to use a 15-minute TTL on its DNS records. I wrote a script that
would tell me how many cache hits I would get if I were using a DNSBL
with a 15-minute TTL. (Perl script appended, but it only works on
Sendmail logs using rsyslog's new RSYSLOG_FileFormat format.)
On our mail server, we received 1449 inbound SMTP connections one day.
Of those, 974 would not have hit the DNS cache, either because they
hadn't been seen before or because 15 minutes had passed since the
last sighting. There were 475 cache hits. So disabling caching
altogether would have increased my load on the authoritative servers
by about 50%.
I took a look at a somewhat busier mail server with 165,149 SMTP
connections / day. If that had been using a DNSBL with a 15-minute TTL,
there would have been only 36,158 hits and 128,991 misses. Doing
authoritative-only queries would increase the load by only ~30%.
In summary, I believe DNS caching is basically *useless* for any site
small enough to use Spamhaus for free. And any very large site is
probably large enough to deserve an rsync feed.
Regards,
David.
# CUT HERE ==
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
my %last_seen;
while() {
my ($y, $m, $d, $h, $min, $sec, $relay) =
/^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).*daemon=MTA,
relay=.*\[(\d+\.\d+\.\d+\.\d+)\]/;
next unless defined($relay);
next if ($relay eq '127.0.0.1');
my $unixtime = timelocal($sec, $min, $h, $d, $m-1, $y);
if (!exists($last_seen{$relay})) {
print MISS for $relay: new entry\n;
$last_seen{$relay} = $unixtime;
} elsif ($unixtime - $last_seen{$relay} 15*60) {
my $secs = $unixtime - $last_seen{$relay};
print MISS for $relay: last seen $secs seconds ago\n;
$last_seen{$relay} = $unixtime;
} else {
print HIT for $relay\n;
# Don't update $last_seen because DNS server would not increase
TTL
}
}
Re: DNS cache efficiency for low-TTL records (was Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01)
Following up on myself... I ran a little experiment. Just for fun, I took a day's worth of logs from a fairly busy server. There were just over 3.1 million SMTP connections/day. If they'd been using a DNSBL with a 15-minute TTL, they would have had about 1.13 million cache misses and 1.97 million cache hits. Turning off caching completely would increase the load on the authoritative server by a factor of about 2.75. This is (to me) surprising. It means you could probably build a DNSBL/WL that permits queries for every single lookup to go to the authoritative servers without terrible difficulty. Scaling up an DNSBL 10x or 100x would be hard, but 3x? Should be doable. (Spamhaus could greatly lower the load on its servers by using much bigger TTLs, especially for lists that don't change often like the PBL. But as another posted mentioned, sometimes DNSBL owners want to see the queries, particularly if they want to charge high-volume users. :) Regards, David.
Re: DNS cache efficiency for low-TTL records (was Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01)
On Tue, Jan 4, 2011 at 9:24 PM, David F. Skoll d...@roaringpenguin.com wrote: (Spamhaus could greatly lower the load on its servers by using much bigger TTLs, especially for lists that don't change often like the PBL. But as another posted mentioned, sometimes DNSBL owners want to see the queries, particularly if they want to charge high-volume users. :) Compared to a typical DNSBL, we have extremely long TTLs for dnswl.org RRs. This does improve caching (and makes it harder to charge the heavy users, yes), but it will also make it harder to correct mistakes. -- Matthias
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
This is a great topic! Is this been discussed at the IETF level? Well, yeah, that's the internet draft that I started this with. There's a parallel discussion in the IETF anti-spam research group (ASRG) which is a better place to continue this. See http://wiki.asrg.sp.am/ which has a link to subscribe to the mailing list. R's, John
Re: DNS cache efficiency for low-TTL records (was Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01)
In summary, I believe DNS caching is basically *useless* for any site small enough to use Spamhaus for free. And any very large site is probably large enough to deserve an rsync feed. Hmmn. See the ASRG list where I've posted some numbers I worked up from my own servers. R's, John
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 01/05/2011 05:14 AM, David F. Skoll wrote: On Tue, 04 Jan 2011 11:01:52 -0500 Rob McEwen r...@invaluement.com wrote: When we are left with only whitelists and no blacklists, an interesting problem will happen... there will be extreme prejudice against ALL new IPs not already whitelisted. Life will become more difficult, but it's not all doom-and-gloom. By default, you should be able to get on the whitelist just by asking. If it turns out you've abused the trust, you get banned for a long time. That's essentially how the Spamhaus Whitelist works. Why focus on DNS IP whitelists? What's wrong with mandated SPF instead? (ie my all ipv6 smtp servers must have explicit SPF records). Much less overhead than RBLs, scales better (we may have near infinite ipv6 addresses, but there will still only be as many DNS domains in ipv6-land as there are in ipv4-land - in the beginning, obviously) I know SPF isn't perfect (we still don't do it ourselves), but ipv6 may change the landscape so much that nothing short of draconian measures may suffice. -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Funny thing, and I think John Levine remembers 1994: OH MY GOD, THE INTERNET WENT COMMERCIAL, with all these new computers, its the end of the internet. and the oft quoted: Breaking Story: Death of the Internet, gif at 11 -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 ISN: 1259*1300 *| *SECNAP Network Security Corporation * Certified SNORT Integrator * 2008-9 Hot Company Award Winner, World Executive Alliance * Five-Star Partner Program 2009, VARBusiness * Best in Email Security,2010: Network Products Guide * King of Spam Filters, SC Magazine 2008 __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ __
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
John Levine said: Rob McEwen said: To be extra clear, the kind of sender's list I was talking about wouldn't be the same as a yellowlist because it would ALL types of IPs (black, white, yellow). Except everyone... including spammers... would have to jump through some hoops to get a single IP that list. But this /then/ VASTLY lowers the number of possible IPs that could be subsequently be whitelisted, blacklisted, or yellowlisted. Depends what your goals are. As soon as you add even the smallest bit of qualification, you have all of the pain of any sort of policy based list, people complaining that they're listed, complaining that they're not listed, lying to you about whether they qualify, and it's not worth the effort. An MTA sees a connecting client as white (accept everything), black (reject everything) or color-to-be-named-later (accept but filter.) As soon as you think a host is not pure spam, you're done, since you'll filter it anyway. John, Please reconsider... and how about this twist... Let the IP registrars (arin.net, etc) add a very nominal fee for allowing networks to designate particular IPs as being used for SMTP. Perhaps something like this: $1.00/year/ip for the non-sequential IPs designated by an organization for SMTP usage $0.25/year/ip for additional /*sequential*/ IPs designated by an organization for SMTP usage (The first IP in a range would cost $1.00, others in the same sequential range would be 25 cents each.) Each IP registrar could then publish a master list of ALL participating IPs, which would be updated once daily, and could be downloaded as a compressed file via HTTP, etc. The ONLY criteria is the payment--and whatever each IP registrar /*already*/ does regarding designating IP ranges for organizations. This would absolutely keep the volume of IPv6 mail-sending IPs under control. Once that is done, 99% of ALL of the problems discussed on this thread disappear! Actually, a private organization could do the same thing... but they'd have two disadvantages over the IP registrars. (1) They'd be accused of taking money from spammers, (2) AND... they'd have a harder time acheiving critical mass. IP registrars already take money from spammers and no one things twice about that. And the IP registrars would have an easier time gaining market share on this idea and could quickly convince everyone in the industry to block e-mail that is not on that master list at the beginning of the spam filtering process. Then the DNSBLs would /only/ have to blacklist particular IPs from amongst those which /are/ included in that master list. ALL other IPs (not on the master list) would be completely be ignored by everything for SMTP, as if those IPs didn't exist. So there wouldn't be a need to blacklist them. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Please reconsider... and how about this twist... Let the IP registrars (arin.net, etc) add a very nominal fee for allowing networks to designate particular IPs as being used for SMTP. Haven't you just reinvented whitelisting? I think it's pretty likely that people will make lists of IPs known to be mail clients to keep down the filtering load, but there's still the problem that bad guys an sign up so you have endless compliance problems. Oh, and if you did that, how would MTAs check that an address of an incoming connection was on the list? That's the issue that started this discussion. Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Not to speak for Rob, but... Haven't you just reinvented whitelisting? I think it's pretty likely that people will make lists of IPs known to be mail clients to keep down the filtering load, but there's still the problem that bad guys can sign up so you have endless compliance problems. I think Rob's idea is that this would be a necessary condition, not a sufficient condition. That is, if you're NOT on this list, no port 25 for you, full stop. If you ARE on this list, you can still be blacklisted, graylisted, content filtered, hit with SMTP prompt delays, and so on. Oh, and if you did that, how would MTAs check that an address of an incoming connection was on the list? That's the issue that started this discussion. If it's an opt-in on a registrar level, I'd think that could be a host file that's rsynced as infrequently as once or twice a day. I mean, as a practical matter your MX records aren't going to propagate through DNS much faster than that anyway. -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com ...Life is not a journey to the grave with the intention of arriving safely in one pretty and well-preserved piece, but to slide across the finish line broadside, thoroughly used up, worn out, leaking oil, and shouting GERONIMO!!! -- Bill McKenna
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/3/2011 9:21 PM, Dave Pooser wrote: Not to speak for Rob, but... Dave, You described my point quite well and I appreciate your help! What I described is vastly different than whitelisting and has massive upsides. I haven't yet found any noteworthy downsides. Overall, this discussion thread took many twists and turns over the past few weeks. And tensions were often very high. But *two things became clear to everyone involved*, even those who disagreed with each other passionately over various things: (1) The problems caused by using IPv6 for SMTP are quite large, and there are *many* interrelated problems... RECAP: dns cache bloating--to a point of ruining ISP's caches, DNSBL bloating, bloating of resource requirements for serving IPv6 DNSBLs distributing the massively larger data files, and dream-come-true scenarios for the spammers--such as giving them the ability to send a million spams each from an individual IP that is never to be heard from again--and that defeats ALL benefits of individual IP blacklisting AND gives the spammers a superb list-washing tool at the same time (In a /*single*/ e-mail campaign, they'll know /*exactly*/ which e-mail addresses are feeding DNSBLs!!!). Some of this was partially solved by blacklisting larger blocks... but no one came up with any good means for blacklisting of larger blocks that doesn't cause collateral damage. OVERALL--THE END RESULT IS AN ABSOLUTE NIGHTMARE!!! (2) These problem are DIRECTLY and ONLY caused by the available pool of IP addresses at a spammer's disposal having grown exponentially with IPv6--so much so that the possible pool of spam-sending IPs is obscenely large--and those IPs will be dolled out in vastly larger numbers than IP allocation under IPv4. Accordingly, my proposed solution VASTLY lowers this number of IPs that could e used for SMTP in a way that is amazingly inexpensive for legit senders, and extremely expensive for spammers who would try to burn through large numbers of IPs. Ironically, as I sat and read all the heated bickering... I was extremely happy that ALL of these problems were finally being openly discussed in a realistic fashion. It is about time. *And thanks, John, for engaging us on this and bringing to light these massive problems!* ...moving on... John Levine said: Haven't you just reinvented whitelisting? No. As described, this is vastly different than whitelisting... and doesn't replace whitelisting in the slightest. Oh, and if you did that, how would MTAs check that an address of an incoming connection was on the list? That's the issue that started this discussion. If it's an opt-in on a registrar level, I'd think that could be a host file that's rsynced as infrequently as once or twice a day. I mean, as a practical matter your MX records aren't going to propagate through DNS much faster than that anyway. Exactly! The master list of IPv6 IPs used for SMTP would only update perhaps even just once per day, and would be available via rsync, ftp, http, etc (in some cases, compressed). BTW - Ironically, it is all the more of an upside that spammers could freely pay registrars for as many IPs to have SMTP designation as desired because, quite frankly, that is a lesser evil than the registrars ever getting political about who gets SMTP-designation. Since this is already an issue they deal with in their regular IP allocations, they probably already handle this quite nicely? In fact, I'd think it is in their best interest to just collect the money and laugh all the way to the bank... and not be bothered with denying SMTP designations, and, in this case, that is a GOOD THING!!! ALSO: With such a system in place, *IPv6 botnet-sent spam via dynamically-assigned IPs connections would be non-existent /by design/ from day one* (that alone is makes this idea worthwhile!) BOTTOM LINE: the IPv6 status quo is a spam sender's dream and a DNSBL's nightmare. My proposed solution is the opposite. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 1/3/11 9:34 PM, Rob McEwen r...@invaluement.com wrote: BTW - Ironically, it is all the more of an upside that spammers could freely pay registrars for as many IPs to have SMTP designation as desired because, quite frankly, that is a lesser evil than the registrars ever getting political about who gets SMTP-designation. Frankly, I'd think that besides costing the spammers money (a good thing in and of itself) it would also be a pretty good spamsign if a block has more than, say, 5 or so registered senders in a /64. Just thinking out loud here -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com Pulling together is the aim of despotism and tyranny. Free men pull in all kinds of directions It's the only way to make progress. -- Terry Pratchett, _The Truth_
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Frankly, I'd think that besides costing the spammers money (a good thing in and of itself) it would also be a pretty good spamsign if a block has more than, say, 5 or so registered senders in a /64. Just thinking out loud here There are a lot of non-spam mail systems with a heck of a lot more than five mail hosts, and I can easily imagine putting a bunch of them on a single LAN where they're be in the same /64. I also don't think it's very realistic to expect that there will be a master mail host file distributed periodically like HOSTS.TXT was. There's a reason that the DNS was invented, and at the time it was, there were a whole lot less hosts on the net than there are mail hosts now. Finally, I presume everyone is aware that there is no possibility whatsoever that RIRs will get into the business of selling mail host IPs, so this entire subthread is 100% hypothetical. R's, John
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 01/04/2011 04:50 PM, Dave Pooser wrote: Frankly, I'd think that besides costing the spammers money (a good thing in and of itself) ...spammers steal other people's resources - so they'll pay nothing... The best case scenario we can ever hope for is that they will be stuck sending all their spam using the From: address and SMTP server of the infected host - nothing better is possible, unless you can figure out how to stop 100% of humanity clicking on %*# executables. This is a great topic! Is this been discussed at the IETF level? This is much bigger than SA. From the sounds of this thread, spam under ipv6 is going to be almost an *infinitely* bigger problem than ipv4. What about some real I want a pony ideas? Mandating SPF/DomainKeys/whatever could be an entirely appropriate response to this - that would be a lot easier than mandating egress filtering/etc (which would never happen - the solution needs to be where the client rejects the server). ie IETF says all ipv6 SMTP sessions must abide by explicit SPF - so we can *reject* anything that doesn't comply, instead of the current ~all lameness we suffer from. Yup, life will be tougher for domains - too bad. -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
Real-world IPv6 allocation policies (was Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01)
Hi, all, We run a system of data collection that collects reputation information about IP addresses. Our system has data on over 18 million IPv4 addresses and 2658 IPv6 addresses (which shows how poor the penetration of IPv6 is.) For details of our system, see http://mimedefang.org/reputation Anyway, I checked to see how many of the IPv6 addresses were in the same /64 and the answer is... a lot of them. All of the 2658 individual addresses are within 1674 different /64s. The average /64 has 1.5 addresses. We've seen as many as 95 individual addresses within the same /64. (And we only see machines that attempt to send mail to one of our sensors. There are probably way more machines in each /64 than what we see.) It seems that many organizations do place multiple machines in the same /64, so /64 granularity may not be good enough for a BL and definitely won't be good enough for a WL. I'm coming to the conclusion that John Levine's proposal or something similar is necessary after all. :( Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Ted Mittelstaedt t...@ipinc.net writes: No, since the number of total host numbers in a /64 is vastly larger than in a /128, if you hold to single number queries then it will blow it out far far faster. This is why I said SA needs to be modified to treat a single hit in a /64 as the entire /64 is contaminated, and cease further queries on numbers in that /64. A /64 is in some respects like an IPv4 /24 except that it has vastly more hosts. Declaring all occupants of a /64 bad due to one misbehaving host seems no more reasonable than blacklisting an entire IPV4 /24 or larger. There are two separate issues here: ability to list individual hosts when the network running the host is basically reasonable but may have compromised hosts. For this, the semantics of the list should be similar to the IPv4 blacklists, but with a larger address space and not necessarily any more entries. ability to aggregate blacklists of prefixes when a spammer is rotating through addresses to evade blacklisting. Here, the BL operator can detect this and publish a blacklist entry for an entire prefix. So a whole /64 or /48 can be blocked with one DNS-published entry, while retaining the ability to not paint all hosts with the same brush. pgpnybOtJdIEH.pgp Description: PGP signature
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Ted Mittelstaedt wrote: End users all over most of the world WANT to interact with foreigners. End users all over the world primarily want to interact with family and friends, 95% of which speak the same language and live in the same country. They DO NOT want to have the Internet on their piece of the the world to become incompatible with everyone else. ASCII and the Latin characters that make it up are the lowest common denominator and everyone's whore and the end users of the world want it that way for e-mail addresses. End users don't care as long as they can write emails and address them to people the way they are used to, i.e. using their local alphabet. They get confused when they can't, but that is of course something you can get used to. /Per Jessen, Zürich
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
And SMTP is the same philosophy. Unicode addressing should rightly be an add-on to a simpler system. And frankly the biggest proponent of EAI is China - and why do you think that this is? Silly me, I thought it was because they have 1.2 billion citizens who read and write Chinese rather than English. In any event, there are better fora for conspiracy theories. I'm just trying to make DNSBLs and DNSWLs work in IPv6. R's, John
Re: Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, Dec 30, 2010 at 12:42 AM, Ted Mittelstaedt t...@ipinc.net wrote: Thus, we can safely make the assumption that any mailserver is going to follow the model of a single host per /64. Thus it will ALSO be just as useful for whitelists to have the same granularity - a /64 - as it would be for blacklists. /64 as a default policy for a whitelist may make sense, but the protocol *must* allow for different granularities as well -- both higher and lower granularities for reasons outlined in another mail (eg shared hosting environments). What this really calls for is a reworking of the SpamAssassin code. SA is going to have to start caching the results of any IPv6 DNS BL queries for a set period of time, probably 2 days. Any IPv6 Cache should observe RRs TTL. address in a BL needs to invalidate all other IPv6 addresses in the /64 that the IPv6 address is in for 2 days. There is no need to do further querying, nor is there a need for the scheme enumerated in the RFC draft. Can you be really, absolutely sure that there will never, ever be a need to report reputation on anything else than /64? -- Matthias
Re: [Asrg] draft-levine-iprangepub-01
On Wed, 29 Dec 2010 15:42:58 -0800 Ted Mittelstaedt t...@ipinc.net wrote: What this really calls for is a reworking of the SpamAssassin code. SA is going to have to start caching the results of any IPv6 DNS BL queries for a set period of time, probably 2 days. Why? Isn't caching the results of queries the job of your name server? Regards, David.
Re: [Asrg] draft-levine-iprangepub-01
On Thu, 30 Dec 2010 10:15:42 +0100 Matthias Leisi matth...@leisi.net wrote: Can you be really, absolutely sure that there will never, ever be a need to report reputation on anything else than /64? I think it's a safe bet, especially for whitelists. If you're whitelisting someone, chances are that person knows what he/she's doing and will use a sensible IP address allocation policy. Actually... is anyone on the list aware of an IPv6 provider that assigns less than a /64 to end-users? My tunnel broker gives us a /64 for our tunnel and a routed /48 for our network. Our hosting provider gives us a /64 for each host. Anyone on the list experience anything different? Regards, David.
Re: [Asrg] draft-levine-iprangepub-01
On 2010/12/30 7:49 AM, David F. Skoll wrote: Actually... is anyone on the list aware of an IPv6 provider that assigns less than a /64 to end-users? My tunnel broker gives us a /64 for our tunnel and a routed /48 for our network. Our hosting provider gives us a /64 for each host. Anyone on the list experience anything different? A /64 is intended to be a host address only and allocations of at least a /56 to individuals are intended but /48's may be preferred to ease routing. My IPv6 plan is to use /48's for everything unless a new practice emerges. -- /Jason smime.p7s Description: S/MIME Cryptographic Signature
IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Hi. I hear there's been some interest in my IPv6 DNSBL proposal. My goal is that since there are (close enough to) no v6 BLs or WLs yet, this is the time to switch to a query design that will scale. The design I put in RFC 5782 isn't it, unfortunately, nor is anything similar to it. We'll have to change our software to handle v6 lookups no matter what, so I don't see it as a big deal whether it's a small change or a slightly larger change. Thus, we can safely make the assumption that any mailserver is going to follow the model of a single host per /64. ... This strikes me as a poor idea for two reasons: it's probably not true, and even if it is, it won't help. The IPv6 address space is big. Very, very big. Even if you chop it in half to /64s, it is still four billion times bigger than the v4 address space. Bad guys hopping around /64s will blow out your DNS cache just as badly as hopping around /128s. And at this point I would not want to assume there is only one host per /64, or that a /64 will contain all good hosts or all bad hosts, since there will doubtless be cases where that's not true. If you've read my proposal (if you haven't, please stop, visit http://tools.ietf.org/html/draft-levine-iprangepub-01 and read it, then come back) you'll see that maintaining a BL/WL is fairly complicated, but the lookups are quite simple. Each lookup involves about five DNS queries, but the design makes it very likely that most if not all of the answers will already be in the local cache, since the queries all start from the top of the same tree. It also ensures that if you do a bunch of lookups to addresses that are near each other, they'll probably all do the same queries, so all after the first will be cached. Another way to look at it is the size of zone: since each DNS record holds 40 entries, the number of records is no more than 1/40th of the size of a record-per-entry design. In the common case that entries span a range of addresses, the number of entries will be even less, a lot less. (Note that rbldnsd synthesizes records on the fly, so that as far as a client can tell, even if the server knows something is a /16, the client sees 64K different records.) And finally, in this design, the client only looks for records that exist, so there should be no negative entries in the cache at all. This tells me that this would have performed better even for short 32 bit addresses. I've given it a fair amount of thought, and I think I have gone through all of the same band-aids everyone else is thinking of, e.g., truncate everything to 64 bits, do some sort of probe to find out the granularity of a range, and they don't work. When you consider the length of the addresses, the number of queries, and the cache behavior, I'm pretty sure this design is vastly better than anything based on the traditional design, and is not an unreasonable amount of work for clients. I don't think it's perfect, and I'd be delighted to get suggestions, but please don't start by assuming that spammers won't be maximally hostile, or that managers will always configure their networks the way you'd prefer. Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 30 Dec 2010 17:13:07 - John Levine jo...@taugh.com wrote: We'll have to change our software to handle v6 lookups no matter what, so I don't see it as a big deal whether it's a small change or a slightly larger change. I agree, so I propose a much larger change: Stop using DNS for this purpose. I don't think it's the right tool for the job. Any protocol that makes lookups in a huge adress space efficient and efficiently-cacheable is going to leak much of the list information. So why not just distribute copies of the entire list in a format that permits efficient lookups and efficient sychronization (eg with rsync)? Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 12:47 PM, David F. Skoll wrote: On 30 Dec 2010 17:13:07 - John Levine jo...@taugh.com wrote We'll have to change our software to handle v6 lookups no matter what, so I don't see it as a big deal whether it's a small change or a slightly larger change. I agree, so I propose a much larger change: Stop using DNS for this purpose. I don't think it's the right tool for the job. Any protocol that makes lookups in a huge adress space efficient and efficiently-cacheable is going to leak much of the list information. So why not just distribute copies of the entire list in a format that permits efficient lookups and efficient sychronization (eg with rsync)? Which leads to another potential problem If blacklists like CBL are currently at 100 MBs (for IPv4)... the bloat for IPv6 could break DNSBLs. RSYNCing Gigabyte (or terabyte!) -sized files is memory and CPU intensive. Loading those into rbldnsd is also resource expensive! Furthermore, getting that data out to DNS mirrors quickly and efficiently is going to be a nightmare! And... this requires that ALL mirrors be upgraded to accommodate the vastly larger size. A better solution (John, I hope you are willing to help with this...) involves some combination of the following three ideas: (1) create a standard whereby non-authenticated IPv6 mail can ONLY be accepted by certain IPs (such as x.x.x.0, if this were an IPv4 rule... translate that to IPv6... perhaps just one designated IP per /48 ??) Any other IP tries to send mail, and it get rejected if it isn't your own user doing SMTP AUTH. Btw - yes, you household appliance can STILL send an email... it will just have to SMTP authenticate to a more valid server first! (2) Why can't Forward Confirmed reverse DNS (FCrDNS) become a standard for IPv6? And we just all agree to reject anything less that this? (sure, some will ignore that... but if enough of us stick together on that... including a few large ISPs... this should gain critical mass!) (3) A shifting of focus on whitelists is important... but some of those shouldn't really be whitelists in the traditional sense. Instead, they should merely indicate that an IP is a candidate for sending mail. Call it an IPv6-sender's list. Don't accept mail unless the sender's IP is on that list... THEN check that IP against an IPv6 blacklist. To get on the generic IPv6-sender's list is easy... but might require (a) FCrDNS, (b) filling out a CAPTCHA-protected form, (c) e-mail verification, using a non-freemail e-mail address, (d) NOT having a non-hidden registration for the domain used in the e-mail address, etc, etc, etc. At the least this prevent a spammer from sending a million spams from a million individual IPs, with each IP never to be seen again--and then bloating IPv6 DNBSLs with useless data! Yes, spammers will /easily/ get on the IPv6-sender's list.. and if that bothers you, you've missed the point! Sure, you can /also/ have REAL whitelists... but they'd serve a different purpose. Now... who would run the IPv6-sender's list? ...I don't know. Even if just individual DNSBLs did this, that would be helpful!! Time is short. If these types of things aren't in an RFC soon, it will be too late. John, please feel free to take any of these ideas and put them in an rfc. No need to give me any credit. I doubt that I'm the first to things of these things anyways! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, 30 Dec 2010 13:19:03 -0500 Rob McEwen r...@invaluement.com wrote: If blacklists like CBL are currently at 100 MBs (for IPv4)... the bloat for IPv6 could break DNSBLs. RSYNCing Gigabyte (or terabyte!) -sized files is memory and CPU intensive. Well, not really... John Levine proposes a way to summarize swaths of IPv6 address space into very little storage, so that shouldn't be an issue. While I'm not crazy about using DNS for this purposes, John's basic ideas are correct. The real problem is the human effort needed to monitor the enormous IPv6 address spave for abuse. I think it'll be hard or impossible to come up with useful and comprehensive IPv6 blacklists. Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 1:26 PM, David F. Skoll wrote: Well, not really... John Levine proposes a way to summarize swaths of IPv6 address space into very little storage, so that shouldn't be an issue. While I'm not crazy about using DNS for this purposes, John's basic ideas are correct. The real problem is the human effort needed to monitor the enormous IPv6 address spave for abuse. I think it'll be hard or impossible to come up with useful and comprehensive IPv6 blacklists. Does John's system do anything to prevent a spammer from sending a million different spams from a million different IPs (one-ip-per-spam) ...with that IP never to be heard from again)? (and with little or zero collateral damage?) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, 30 Dec 2010 13:34:16 -0500 Rob McEwen r...@invaluement.com wrote: Does John's system do anything to prevent a spammer from sending a million different spams from a million different IPs (one-ip-per-spam) ...with that IP never to be heard from again)? Well, obviously not. Nothing can control what a spammer does. John's system ensures that *if* a spammer does that, *then* you don't blow your DNS servers' caches (assuming the blacklist operator has constructed the blacklist properly.) Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, 30 Dec 2010, David F. Skoll wrote: On 30 Dec 2010 17:13:07 - John Levine jo...@taugh.com wrote: We'll have to change our software to handle v6 lookups no matter what, so I don't see it as a big deal whether it's a small change or a slightly larger change. I agree, so I propose a much larger change: Stop using DNS for this purpose. I don't think it's the right tool for the job. Any protocol that makes lookups in a huge adress space efficient and efficiently-cacheable is going to leak much of the list information. So why not just distribute copies of the entire list in a format that permits efficient lookups and efficient sychronization (eg with rsync)? Timeliness? How often are you going to refresh the local copy of the entire WL/BL? Or are you assuming the WL/BL will be relatively unchanging over time? Overall bandwidth? How big is the overall WL/BL? Can hosting of the file be as efficiently distributed across multiple caching hosts (e.g. via Coral) as can DNS? What's the download volume vs. the DNS query volume? Or are you assuming the refresh protocol supports incremental updates? Does rsync support any incremental update mechanism other than appending? That would work for added entries, how do you delete? Local spam volume vs. the size of the full BL? If I only get a hundred spams a day is it reasonable for me to store the full BL locally? Perhaps several BLs? Are you, essentially, proposing the replacement of DNS with /etc/hosts? Granted, rsync or something similar may be a better solution than DNS in some cases, but I think it's unwise to completely discard it. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Gun Control laws cannot reduce violent crime, because gun control laws focus obsessively on a tool a criminal might use to commit a crime rather than the criminal himself and his act of violence. --- 22 days since the first successful private orbital launch (SpaceX)
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
I agree, so I propose a much larger change: Stop using DNS for this purpose. I don't think it's the right tool for the job. Sigh. Yes, that's one of the bad ideas. Remember that part of the goal is to keep the traffic to and from the DNSBL/WL's servers under control. Any protocol that makes lookups in a huge adress space efficient and efficiently-cacheable is going to leak much of the list information. So why not just distribute copies of the entire list in a format that permits efficient lookups and efficient sychronization (eg with rsync)? If you're familiar with populare DNSBLs, this is not a new idea. Spamhaus, for example, provides low volume queries for free, medium volume queries for a charge, and rsync access for a higher charge. See: http://www.spamhaustech.com/datafeed/index.lasso Consider the amount of traffic involved in doing an rsync: set up a TCP session, do the key exchange to set up a SSH session, then start comparing pieces and checksums to figure out which parts of the files need to be transferred, that's a fair bit of traffic, particularly for a large zone file. The tradeoff point where it's cheaper than doing queries is quite high. If you've got a giant mail system, it makes sense, but if you have one or two MTAs, even fairly busy ones, it doesn't. Hence my goal to concoct something which allows efficient publication and caching via the DNS. Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, 30 Dec 2010 10:36:59 -0800 (PST) John Hardin jhar...@impsec.org wrote: Timeliness? How often are you going to refresh the local copy of the entire WL/BL? Or are you assuming the WL/BL will be relatively unchanging over time? A WL should be relatively unchanging over time. I doubt BLs will be useful in IPv6, but even assuming they are, doing an update every 10-20 minutes shouldn't be a problem. (You don't have to use rsync... it was just an example. I'm sure it's possible to come up with a protocol targeted for more efficient updates given the problem domain.) Overall bandwidth? How big is the overall WL/BL? Can hosting of the file be as efficiently distributed across multiple caching hosts (e.g. via Coral) as can DNS? All of these problems seem to be managed quite nicely by projects such as ClamAV, which distributes virus signatures. (AFAIK, ClamAV does use a purpose-built update format that lets you download incremental changes quite efficiently.) [...] Are you, essentially, proposing the replacement of DNS with /etc/hosts? Not necessarily, but I am saying that DNS was never designed for WL/BL purposes and it may be time to replace it with something better if we're going to make major changes anyway. [Aside: while John's proposal will help prevent DNS-based blacklist lookups from blowing server caches, it doesn't prevent said caches from blowing up anyway as MTAs do reverse-lookups on inbound connections. If you do any kind of lookup at all on inbound SMTP connections, an attacker can cause trouble for you, especially if he controls the reverse DNS space for a /64. :( All of this argues that granularity for blacklists, firewall rules, etc. is going to be a /64 or bigger in reality.] Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 30 Dec 2010 18:43:50 - John Levine jo...@taugh.com wrote: I agree, so I propose a much larger change: Stop using DNS for this purpose. I don't think it's the right tool for the job. Sigh. Yes, that's one of the bad ideas. What is? Using DNS or using something else? :) [...] Consider the amount of traffic involved in doing an rsync: I used rsync as an example. You can use a more efficient technique; I gave ClamAV's signature-distribution mechanism as an example of a system that works pretty well. Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
If blacklists like CBL are currently at 100 MBs (for IPv4)... the bloat for IPv6 could break DNSBLs. RSYNCing Gigabyte (or terabyte!) -sized files is memory and CPU intensive. Loading those into rbldnsd is also resource expensive! Furthermore, getting that data out to DNS mirrors quickly and efficiently is going to be a nightmare! And... this requires that ALL mirrors be upgraded to accommodate the vastly larger size. Right. I don't think the CBL will get much larger, since it will certainly do /64 granularity, but it'll still be a challenge to query efficiently. (1) create a standard whereby non-authenticated IPv6 mail can ONLY be accepted by certain IPs (such as x.x.x.0 Sorry, no chance. (2) Why can't Forward Confirmed reverse DNS (FCrDNS) become a standard for IPv6? Because rDNS lookups will explode your cache just as badly as DNSBL lookups. In the words of a friend who used to run a very large mail system, when I asked him about IPv6 rDNS: Just Say No. rDNS isn't likely to be useful at all for v6, although you could try something like CSV based on looking up the EHLO name. (3) A shifting of focus on whitelists is important... but some of those shouldn't really be whitelists in the traditional sense. Instead, they should merely indicate that an IP is a candidate for sending mail. This one I agree with. The Spamhaus whitelist is intended only for very virtuous sources of mail, but it will clearly also be useful to have what was called a yellow list a few days ago, hosts that send enough real mail that you can't just blacklist them even if you see some spam. R's, John
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
I used rsync as an example. You can use a more efficient technique; I gave ClamAV's signature-distribution mechanism as an example of a system that works pretty well. Hey! I have an idea! How about if we form the data into a B-tree and let people download pages on demand via the DNS? R's, John
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 30 Dec 2010 18:57:44 - John Levine jo...@taugh.com wrote: Hey! I have an idea! How about if we form the data into a B-tree and let people download pages on demand via the DNS? Nah, I have a better idea... a B-ish tree where some nodes can get out of sync because of caching. Won't be a problem in practice. Wait: We'll lower the TTL really low so this isn't a problem! And that will not hurt caching efficiency! Or we'll add a bunch of CNAMEs to partly patch up the problem... won't be an issue on a frequently-updated list, for sure. John, I agree that your draft is clever. But I think it's really stretching DNS way beyond what it was designed for and it might be time to look at a different approach. To paraphrase the old saying, when all you have is DNS, every problem looks like a lookup. Reagrds, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
(Sorry, sent to David only by error) On Thu, Dec 30, 2010 at 8:05 PM, Matthias Leisi matth...@leisi.net wrote: On Thu, Dec 30, 2010 at 7:26 PM, David F. Skoll d...@roaringpenguin.com wrote: The real problem is the human effort needed to monitor the enormous IPv6 address spave for abuse. I think it'll be hard or impossible to come up with useful and comprehensive IPv6 blacklists. Some types may still be feasible, eg something à la Spamhaus PBL (nothing in this IPv6-/32 should send direct-to-MX). -- Matthias
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
(Same error on this mail, I should pay more attention to To: and the reply button. Sorry for the mess) On Thu, Dec 30, 2010 at 8:10 PM, Matthias Leisi matth...@leisi.net wrote: On Thu, Dec 30, 2010 at 7:43 PM, John Levine jo...@taugh.com wrote: Any protocol that makes lookups in a huge adress space efficient and efficiently-cacheable is going to leak much of the list information. As an operator of a whitelist, I don't care too much about this. Yes, theoretically someone could get our data by doing enough queries to public nameservers. But I doubt it will be worth the effort for the attacker: he would keep doing this over and over again to keep up with the changes, and would sooner or later be blocked due to too high traffic on the public nameservers. a large zone file. The tradeoff point where it's cheaper than doing queries is quite high. If you've got a giant mail system, it makes sense, but if you have one or two MTAs, even fairly busy ones, it doesn't. I believe (although I haven't thought it through) that this largely depends on the amount of changes in the list data. At dnswl.org, our data changes only slowly. Rsync transfers are very efficient in terms of bandwidth, but CPU intensive nevertheless. -- Matthias
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 2:09 PM, David F. Skoll wrote: But I think it's really stretching DNS way beyond what it was designed for and it might be time to look at a different approach. But David, every example you've provided requires vastly more resources then blocking a spam with a single DNS lookup to rbldnsd. Heck, even a dozen separate DNSBL lookups against a local rbldnsd server is order of magnitudes faster and less resource intensive than accepting the entire message, and running that message against ClamAv (one of your examples). Many large ISPs simply will not ever spend that much $$/message. Otherwise, you'd have to convince the CEO of Comcast to increase their IT budget by 100x... and that would cut into profits... and he'd be fired by the board for that. (to give just one example) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
(3) A shifting of focus on whitelists is important... but some of those shouldn't really be whitelists in the traditional sense. Instead, they should merely indicate that an IP is a candidate for sending mail. This one I agree with. The Spamhaus whitelist is intended only for very virtuous sources of mail, but it will clearly also be useful to have what was called a yellow list a few days ago, hosts that send enough real mail that you can't just blacklist them even if you see some spam. FWIW, that's similar to what dnswl.org is doing with the none, low, med and hi scores. -- Matthias PS: Rob, I still have your mail to us at dnswl.org in our request queue, it has not been forgotten ;)
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 1:55 PM, John Levine wrote: it will clearly also be useful to have what was called a yellow list a few days ago, hosts that send enough real mail that you can't just blacklist them even if you see some spam. John, First, let me mention that I'm grateful that you are working on this! We certainly need your help! To be extra clear, the kind of sender's list I was talking about wouldn't be the same as a yellowlist because it would ALL types of IPs (black, white, yellow). Except everyone... including spammers... would have to jump through some hoops to get a single IP that list. But this /then/ VASTLY lowers the number of possible IPs that could be subsequently be whitelisted, blacklisted, or yellowlisted. And even though you mentioned that FCrDNS wouldn't work as a spam filtering defense... things like FCrDNS could still be of used as a criteria for entry into this master IPv6 sender's list (as a means to keep the volume further under control.) -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, 30 Dec 2010 14:18:13 -0500 Rob McEwen r...@invaluement.com wrote: On 12/30/2010 2:09 PM, David F. Skoll wrote: But I think it's really stretching DNS way beyond what it was designed for and it might be time to look at a different approach. But David, every example you've provided requires vastly more resources then blocking a spam with a single DNS lookup to rbldnsd. That's true... for IPv4. For IPv6, when a spammer can easily command 2^64 separate addresses or more, then even very clever techniques require on the order of 5 lookups. And (at least in John's proposal) there's a serious tension between update frequency and ensuring that the B-tree structure is consistent. If you have a local database on-disk, you can do a lookup extremely quickly. By definition, a local database has to be at least as fast as a DNS lookup because when you make a DNS query, the DNS server has to do a lookup in *it's* local database. Now obviously, there's a breakpoint at which synchronizing the local database from the master becomes cheaper than doing lookups. Right now, that's quite high, but it will move lower with IPv6. Heck, even a dozen separate DNSBL lookups against a local rbldnsd server is order of magnitudes faster and less resource intensive than accepting the entire message, and running that message against ClamAv (one of your examples). I gave ClamAV as an example of efficient distribution of a large and frequently-updated database. I in no way implied that we should abandon IP address lookups in favour of only content-scanning. Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 2:28 PM, David F. Skoll wrote: I in no way implied that we should abandon IP address lookups in favour of only content-scanning Thanks for the clarification! -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
John, I agree that your draft is clever. But I think it's really stretching DNS way beyond what it was designed for and it might be time to look at a different approach. To paraphrase the old saying, when all you have is DNS, every problem looks like a lookup. To be honest, my first reaction to the proposal was similar. Additionally, I'm a bit worried by the complexity we add to a previously extremely simple protocol. From my perspective as an operator of a whitelist, I have three main concerns: 1) I want to be able to manage the load on my (public, for-free) infrastructure. 2) I want to make it easy for filters to use our data (both in development and operations). 3) I want to get some insight into what is being queried (to identify good [and bad] e-mail senders we don't know about yet). John's proposal should help me with #1, and possibly with #2, since it is mostly an evolution of existing concepts and tool chains. Unfortunately, #3 will get much harder. -- Matthias
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
To be extra clear, the kind of sender's list I was talking about wouldn't be the same as a yellowlist because it would ALL types of IPs (black, white, yellow). Except everyone... including spammers... would have to jump through some hoops to get a single IP that list. But this /then/ VASTLY lowers the number of possible IPs that could be subsequently be whitelisted, blacklisted, or yellowlisted. Depends what your goals are. As soon as you add even the smallest bit of qualification, you have all of the pain of any sort of policy based list, people complaining that they're listed, complaining that they're not listed, lying to you about whether they qualify, and it's not worth the effort. An MTA sees a connecting client as white (accept everything), black (reject everything) or color-to-be-named-later (accept but filter.) As soon as you think a host is not pure spam, you're done, since you'll filter it anyway. And even though you mentioned that FCrDNS wouldn't work as a spam filtering defense... No, no, rDNS won't work at all, for anything. It has the same problem as naive DNSBLs, blows out the cache when bad guys use lots of addressess. You don't even dare do the lookups. Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY I dropped the toothpaste, said Tom, crestfallenly.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
John, I agree that your draft is clever. But I think it's really stretching DNS way beyond what it was designed for and it might be time to look at a different approach. To paraphrase the old saying, when all you have is DNS, every problem looks like a lookup. I agree that it's sort of an odd way to use the DNS. But the DNS has a huge advantage over hypothetical alternatives -- the DNS exists, and the alternatives don't. Consider all of the cruddy middleware that has special cases to let DNS traffic through, the extreme efficiency of DNS queries, and the universal availability of DNS caches. Before I switched to an alternative I would want to make really sure that when I was done I would end up with something that actually worked better. I'm not wedded to the CNAME hack. Maybe some sort of version number that would give the client a hint to go back and start over would work better. Or quite possibly the CNAMEs are adequate to keep clients no more than a few minutes out of sync with the server, which is all BLs expect now. Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY I dropped the toothpaste, said Tom, crestfallenly. PS: While you're at it, SMTP needs to be replaced, too.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 9:13 AM, John Levine wrote: Hi. I hear there's been some interest in my IPv6 DNSBL proposal. My goal is that since there are (close enough to) no v6 BLs or WLs yet, this is the time to switch to a query design that will scale. The design I put in RFC 5782 isn't it, unfortunately, nor is anything similar to it. We'll have to change our software to handle v6 lookups no matter what, so I don't see it as a big deal whether it's a small change or a slightly larger change. Thus, we can safely make the assumption that any mailserver is going to follow the model of a single host per /64. ... This strikes me as a poor idea for two reasons: it's probably not true, and even if it is, it won't help. The IPv6 address space is big. Very, very big. Even if you chop it in half to /64s, it is still four billion times bigger than the v4 address space. Bad guys hopping around /64s will blow out your DNS cache just as badly as hopping around /128s. No, since the number of total host numbers in a /64 is vastly larger than in a /128, if you hold to single number queries then it will blow it out far far faster. This is why I said SA needs to be modified to treat a single hit in a /64 as the entire /64 is contaminated, and cease further queries on numbers in that /64. And at this point I would not want to assume there is only one host per /64, or that a /64 will contain all good hosts or all bad hosts, since there will doubtless be cases where that's not true. Well for starters it almost sounds to me like your not that familiar with IPv6 to even say this. The lower 64 bits of the address is the interface identifier, and the upper 64 bits is the sub-network prefix. It is extremely abnormal for a host to change it's MAC address every few milliseconds, so the idea that a spammer could cycle through the lower /64 using 1 address per message is in the realm of extreme improbability. In my opinion you are arguing from a paradigm of the politics of scarcity used in IPv4. Having more potential numbers does not mean that the number of spammers will magically increase. There will be the same number of spammers as there have always been. The only reason to run around saying the sky is falling is if your coming at this from the point of view that it would be a normal thing to see packets from the same /64 that have many different interface identifiers, which there is no logical need for this to ever happen. If you've read my proposal (if you haven't, please stop, visit http://tools.ietf.org/html/draft-levine-iprangepub-01 and read it, then come back) you'll see that maintaining a BL/WL is fairly complicated, but the lookups are quite simple. Each lookup involves about five DNS queries, but the design makes it very likely that most if not all of the answers will already be in the local cache, since the queries all start from the top of the same tree. It also ensures that if you do a bunch of lookups to addresses that are near each other, they'll probably all do the same queries, so all after the first will be cached. Another way to look at it is the size of zone: since each DNS record holds 40 entries, the number of records is no more than 1/40th of the size of a record-per-entry design. In the common case that entries span a range of addresses, the number of entries will be even less, a lot less. (Note that rbldnsd synthesizes records on the fly, so that as far as a client can tell, even if the server knows something is a /16, the client sees 64K different records.) And finally, in this design, the client only looks for records that exist, so there should be no negative entries in the cache at all. This tells me that this would have performed better even for short 32 bit addresses. I've given it a fair amount of thought, and I think I have gone through all of the same band-aids everyone else is thinking of, e.g., truncate everything to 64 bits, do some sort of probe to find out the granularity of a range, and they don't work. wrong. You are just enamored with the idea of over engineering this new paradigm you have dreamed up rather than seriously looking at the rather easy and simple assumptions that can be made to make the existing paradigm work with IPv6. There is no reason that treating a /64 in IPv6 will not work the same as treating a /32 works in IPv4. When you consider the length of the addresses, the number of queries, and the cache behavior, that is hand-waving. I'm pretty sure this design is vastly better than anything based on the traditional design, and is not an unreasonable amount of work for clients. You have started with the assumption that the existing design is fundamentally flawed and that ANY new replacement design is going to be better no matter how half-baked it is. You would be better off starting with the assumption that the existing design is fine, but that it needs adjusting for the new circumstances for IPv6. Why don't you write an RFC
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Now obviously, there's a breakpoint at which synchronizing the local database from the master becomes cheaper than doing lookups. Right now, that's quite high, but it will move lower with IPv6. Why do you say that? The number of computers on the net isn't going to be much bigger with IPv6. They're just spread out in a much larger address space. We also seem to have a diagreement about BL usage patterns. The BLs I know have a smallish number of very heavy clients and then a long tail of clients that get smaller and smaller. It makes sense for the biggest ones to keep their own mirrors, but there's a whole lot of small ones, each of whom will never look at more than a small fraction of the data, although their aggregate traffic is substantial. So if you say that everyone has to maintain a mirror to get a BL's data, you're saying that small clients can't use BLs at all. Is that realistic? Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
Ah, I see the problem. You're assuming that spammers will follow the rules. That's a poor assumption. The IPv6 address space is big. Very, very big. Even if you chop it in half to /64s, it is still four billion times bigger than the v4 address space. Bad guys hopping around /64s will blow out your DNS cache just as badly as hopping around /128s. No, since the number of total host numbers in a /64 is vastly larger than in a /128, if you hold to single number queries then it will blow it out far far faster. I suppose that's technically correct, in the sense that blowing out in a millisecond is faster than blowing it out in a minute, but that hardly matters to people running DNS caches. Let's do a thought experiment: imagine you have a big honking DNS cache with 100 billion slots. If we had a BL with one potential entry per /64 (keeping in mind that a cache remembers both successful and failed queries), how much of the address space can you query before your cache fills up? Answer 0.0054% Kaboom! Well for starters it almost sounds to me like your not that familiar with IPv6 to even say this. The lower 64 bits of the address is the interface identifier, and the upper 64 bits is the sub-network prefix. If you use SAA, sure. If you use DHCP, the address layout is whatever it is. It is extremely abnormal for a host to change it's MAC address every few milliseconds, so the idea that a spammer could cycle through the lower /64 using 1 address per message is in the realm of extreme improbability. Like I said, you're making the poor assumption that spammers will follow your rules. In reality, they'll do whatever they think will get their spam through. The only reason to run around saying the sky is falling is if your coming at this from the point of view that it would be a normal thing to see packets from the same /64 that have many different interface identifiers, which there is no logical need for this to ever happen. Like I said, you're making the poor assumption that spammers will follow your rules. In reality, they'll do whatever they think will get their spam through. You would be better off starting with the assumption that the existing design is fine, but that it needs adjusting for the new circumstances for IPv6. I did that. It doesn't work. R's, John
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 30 Dec 2010 17:49:46 -0500 John R Levine jo...@taugh.com wrote: [...] I'm not wedded to the CNAME hack. Actually, I was thinking about that. Consider a hack on a DNS server that gives all records an absolute expiry time that marches forward in (say) 5-minute intervals. Then when the DNS server is queried, the TTL is computed to be the difference between the current time and the next absolute expiry. In that way, you can try to expire all related records at close to the same time. Just an idea. PS: While you're at it, SMTP needs to be replaced, too. Apples and oranges. SMTP was designed for sending email, which it excels at. The DNS was designed as essentially a distributed lookup table. It was never designed to be warped into a read-only B-tree. :) Regards, David. PS: Alternatives do exist. ClamAV's signature-distribution mechanism is one.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 31 Dec 2010 01:19:16 - John Levine jo...@taugh.com wrote: Now obviously, there's a breakpoint at which synchronizing the local database from the master becomes cheaper than doing lookups. Right now, that's quite high, but it will move lower with IPv6. Why do you say that? The number of computers on the net isn't going to be much bigger with IPv6. They're just spread out in a much larger address space. It's pretty reasonable to assume that when spammers can abuse having huge address spaces at their disposal, they will. So it would make sense for them to crank up their sending volume and send from multiple addresses, especially if they can determine that some attempts have been rejected by recipients. Ie, if dead::beef fails, it can't hurt to try dead::bef0, etc. [...] So if you say that everyone has to maintain a mirror to get a BL's data, you're saying that small clients can't use BLs at all. I'm not saying that at all... why would you think that? Again, to use my favourite example, thousands of tiny sites don't have any problems with maintaining ClamAV signature databases. So why would they have problems maintaining BL/WL data, especially if convenient freshclam-analagous software became available? Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 5:43 PM, John Levine wrote: Ah, I see the problem. You're assuming that spammers will follow the rules. That's a poor assumption. No, I am assuming the spammers will do as they have always done in the past - attempt to use other people's computers for free. Other computers that are NOT cycling through lots of IP number in the normal case. The IPv6 address space is big. Very, very big. Even if you chop it in half to /64s, it is still four billion times bigger than the v4 address space. Bad guys hopping around /64s will blow out your DNS cache just as badly as hopping around /128s. No, since the number of total host numbers in a /64 is vastly larger than in a /128, if you hold to single number queries then it will blow it out far far faster. I suppose that's technically correct, in the sense that blowing out in a millisecond is faster than blowing it out in a minute, but that hardly matters to people running DNS caches. Let's do a thought experiment: imagine you have a big honking DNS cache with 100 billion slots. If we had a BL with one potential entry per /64 (keeping in mind that a cache remembers both successful and failed queries), how much of the address space can you query before your cache fills up? Answer 0.0054% Kaboom! Which is precisely why the querying app, -SA-, needs to get smarter about doing this and limit their queries. Don't you realize that the same thing could be done TODAY to limit queries? SA's developers could make a very (IMHO) legitimate assumption that any IPv4 address that comes up as a positive hit on a BL automatically marks the entire /24 that the IPv4 address is in, based on the idea that a spammer is going to obtain a /24 subnet and cycle through the IP numbers in that subnet. Or it could get smarter and when a BL hit comes though, it could query the RIR's whois database, parse out the subnet that the IPv4 address is in, and blacklist that subnet. The only reason nobody has done this is because there has been no interest in limiting DNS queries from most SA users. The other thing is that if a spammer blows out a client's DNS cache (or a client's ISP's dns cache) then the MTA is going to start hanging, while it waits for DNS queries that never return (since the server is overloaded) and mail receiving will get very, very slow - hardly the situation the spammer wants when their intent is to deliver e-mail spam!!! Any attempt to roll through adjacent IPv6 numbers from a /64 or even from adjacent /64's is a condition that the RBL can easily check for with just a little bit of logic added, and this kind of behavior should instantly mark the e-mail sender as malevolent and a likely spammer. The spammers intent is to camouflage itself, to appear as similar as possible to other legitimate mail senders. Legitimate mail senders will not be rolling through all IPv6 addresses in a /64 so if a spammer wants to stay below the radar and not get noticed then they won't be doing it either. I live in a neighborhood in a house with a large window that faces the street. While it is in theory possible that a gang banger may come through the neighborhood indiscriminately shooting at homes, I am not going to be replacing that window with bulletproof glass. Just because it is possible for the window to be shot out doesn't mean it is ever going to happen. And just because a spammer may be able to roll through individual /128's in a /64 does not mean that any of them ever is going to do it. After all, just because a spammer can get a real job and quit living in their mothers basement doesn't mean that any of them are ever going to do that, either. Well for starters it almost sounds to me like your not that familiar with IPv6 to even say this. The lower 64 bits of the address is the interface identifier, and the upper 64 bits is the sub-network prefix. If you use SAA, sure. If you use DHCP, the address layout is whatever it is. Huh? You do realize there's 2 different DHCP's under IPv6, right? awww screw it, I'm talking to a wall, here. It is extremely abnormal for a host to change it's MAC address every few milliseconds, so the idea that a spammer could cycle through the lower /64 using 1 address per message is in the realm of extreme improbability. Like I said, you're making the poor assumption that spammers will follow your rules. In reality, they'll do whatever they think will get their spam through. and doing this cycling just makes them much more visible as a spammer a lot quicker, thus making it a lot easier to smack 'em down faster. The only reason to run around saying the sky is falling is if your coming at this from the point of view that it would be a normal thing to see packets from the same /64 that have many different interface identifiers, which there is no logical need for this to ever happen. Like I said, you're making the poor assumption that spammers will follow your rules. In reality, they'll do
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, Dec 30, 2010 at 5:21 PM, Ted Mittelstaedt t...@ipinc.net wrote: On 12/30/2010 5:43 PM, John Levine wrote: Ah, I see the problem. You're assuming that spammers will follow the rules. That's a poor assumption. No, I am assuming the spammers will do as they have always done in the past - attempt to use other people's computers for free. Other computers that are NOT cycling through lots of IP number in the normal case. I didn't want to get into this debate, but I think this point is naively optimistic. If a system is capable of cycling through IP addresses, the spammer will take advantage of this. It is trivial to do this on a Linux machine without disrupting operation of the owner's software by adding/removing IP aliases. I would assume there is a way to do it on Windows as well, although it is better hidden. Warren
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On Thu, 30 Dec 2010 19:21:25 -0800 Ted Mittelstaedt t...@ipinc.net wrote: No, I am assuming the spammers will do as they have always done in the past - attempt to use other people's computers for free. Other computers that are NOT cycling through lots of IP number in the normal case. That's because they can't. Most end-user computers won't get very far if they attempt to use an IP address other than the provider-assigned ones. Things change with IPv6. You can typically pick from 2^64 possible addresses per machine without any restrictions from your provider. [...] Don't you realize that the same thing could be done TODAY to limit queries? SA's developers could make a very (IMHO) legitimate assumption that any IPv4 address that comes up as a positive hit on a BL automatically marks the entire /24 that the IPv4 address is in, based on the idea that a spammer is going to obtain a /24 subnet and cycle through the IP numbers in that subnet. That's a very bad assumption. We use a colocated server as our ourbound mail host and it's assigned an IPv4 /29. There could be up to 32 completely-unrelated machines in our /24; blacklisting us because of one of them would be very unfair. [...] The only reason nobody has done this is because there has been no interest in limiting DNS queries from most SA users. No. People don't do that because it's far too draconian. The other thing is that if a spammer blows out a client's DNS cache (or a client's ISP's dns cache) then the MTA is going to start hanging, while it waits for DNS queries that never return (since the server is overloaded) and mail receiving will get very, very slow - hardly the situation the spammer wants when their intent is to deliver e-mail spam!!! Umm... and that's OK? Well for starters it almost sounds to me like your not that familiar with IPv6 to even say this. The lower 64 bits of the address is the interface identifier, and the upper 64 bits is the sub-network prefix. It doesn't have to be: $ host mail.ipv6.roaringpenguin.com mail.ipv6.roaringpenguin.com has IPv6 address 2607:f748:1200:fb:70:38:112:54 70:38:112:54 bears no relationship to any MAC address on that machine. (But check out the IPv4 adress of mail.roaringpenguin.com) and doing this cycling just makes them much more visible as a spammer a lot quicker, thus making it a lot easier to smack 'em down faster. So assume a spammer has 1,000 botnet nodes, each of which has 2^64 possible IPv6 addresses. Explain how you can efficiently detect such cycling and block it. Perhaps you've heard of snowshoe spamming? It's far better for the RBL's to just blacklist the entire /48 that a spamming IPv6 address appears in. Sure that sounds draconian but that's because your thinking in IPv4 address-scarcity terms. The RBLs can always offer 3 different query servers, one for /48's, one for /56's and one for /64s but a /48 is what we need to be shooting for. I think /48 might be a bit much, but here I mostly agree with you. I think John's solution is over-engineered and that a /64 or greater granularity would be perfectly fine. Regards, David.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 8:10 PM, David F. Skoll wrote: So assume a spammer has 1,000 botnet nodes, each of which has 2^64 possible IPv6 addresses. Explain how you can efficiently detect such cycling and block it. Well perhaps not efficiently but the RBL has got to step up to the plate and do some more analysis of IPv6 addresses on received spam Each IPv6 botnet will be originating from at most a /48 If an RBL gets 3 or 4 spam reports from IPv6 addresses within that /48 then it ought to just go ahead and blacklist every IP in the /48 Yes I know there will be collateral damage in cases where the ISP is being overly miserly with it's IPv6. But such ISP's need to get off the damn can and read RFC 3177. Perhaps you've heard of snowshoe spamming? This brings up another point with RBLs that is being ignored in the discussion. Assume for the sake of argument that under IPv6 this so-called snowshoe spamming becomes very prevalent WITHIN smaller IPv6 blocks like /64's and /48's. As we all know most RBL's operate off of honeypot and abandoned e-mail addresses that are on spammers lists. If under IPv6 an RBL continues to adhere to the principle that every IP number is legitimate until it proves itself bogus then it is clear that a spammer can merely rotate through every possible IPv6 address in a small block of IPv6 and end up with maybe a couple dozen RBL listings WITHIN the block he is using, that will never be hit again. Obviously the RBL is not doing anyone a service in that case. I just cannot see how the RBL's can continue in this manner, they have absolutely got to introduce some logic into their listings that starts to consider a minimum allocation and Spam Assassin needs to do the same thing. It's far better for the RBL's to just blacklist the entire /48 that a spamming IPv6 address appears in. Sure that sounds draconian but that's because your thinking in IPv4 address-scarcity terms. The RBLs can always offer 3 different query servers, one for /48's, one for /56's and one for /64s but a /48 is what we need to be shooting for. I think /48 might be a bit much, but here I mostly agree with you. I think John's solution is over-engineered and that a /64 or greater granularity would be perfectly fine. It will only work when SA matches this logic which is why an RFC is called for that defines a minimum contaminated block of IPv6 -or at least an agreement between the RBL's and the spamfilters. If a spammer is rotating though IPv6 numbers then a site running SA is going to start generating a lot of queries and the logical way to put a stop to this is for SA to maintain an internal database and query that first, before querying the RBL servers. If an IP is present anywhere in a /48 within the internal DB of SA then bang - it's spam. No need to query an RBL. Conversely if the RBL recieves a spam then after extracting the IPv6 address from it then bang - the entire /48 is blacklisted that that IP came from. Then if the RBL gets a query for any IP within that /48 block then bang- it's spam. RFC3177 should be the guide, here. Yes, there are a lot more /48's out there but clearly we have a problem with ISP's who host snowshoe spammers out there. This problem can be taken care if by the RBL's getting a lot more militant - and if they see a pattern of /48's from a particular ISP then BL the entire /32 One thing that will help here is that the RIR's have all taken the initiative to assign very large minimum allocations. It will no longer be a world where an ISP can go back repeatedly to the RIR and request small block after small block, of fresh IP numbers to subnet out to snowshoe spammers. Nowadays the ISP gets a massive /32 and since there will be so few ISP's who LEGITIMATELY need successive /32's we will not see the issue as much where a single ISP has blocks scattered throughout the IP space that it can use to help conceal showshoers. The RIR's are doing this to help reduce the DFZ but it has other benefits like this. And the very large ISP's are coming out of the gate now with far more massive IPv6 allocations than a /32. Lastly it is important to keep in mind that EVERY IPv6 address assigned is going to an org that PAYS A FEE to an RIR. At least now with ARIN (I don't know about the other RIR's) ARIN is now REGULARLY checking up on bogus WHOIS entries. I was one of the people who helped push that though, by the way. Since these ISP's are paying a fee they had to sign a contract with the RIR that has the usual statements in it that disallow fraud. Under the IPv6 regime if a spamfighter sees a correlation on a /32 of a high amount of spam coming from it, and starts investigating and discovers that the street address/e-mail address of the ISP that is assigned the /32 is bogus then he can report the ISP to the RIR and the RIR will sue them for breech of contract and pull their allocation. That was not possible under the IPv4 regimen where the legacy allocations had no
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
I'm not wedded to the CNAME hack. Actually, I was thinking about that. Consider a hack on a DNS server that gives all records an absolute expiry time that marches forward in (say) 5-minute intervals. Then when the DNS server is queried, the TTL is computed to be the difference between the current time and the next absolute expiry. That had occurred to me. Another possibility is to embed serial numbers in the records and if the client sees it's out of sync, it goes back to the root and starts over. PS: While you're at it, SMTP needs to be replaced, too. Apples and oranges. SMTP was designed for sending email, which it excels at. The DNS was designed as essentially a distributed lookup table. It was never designed to be warped into a read-only B-tree. :) Snerk. SMTP was designed for a network with no security where everyone behaved themselves and all the mail was ASCII text. We shoehorned in formatted mail and file attachments with MIME, kludged in some security with S/MIME and later SPF and DKIM, and are now in the midst of a really, really big kludge to try to add Unicode addressing in EAI. It passed its best-by date decades ago, but it shares with the DNS the fact that it exists, and the putatively better alternatives don't.* Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY I dropped the toothpaste, said Tom, crestfallenly. * - Well, OK, X.400 exists. Sort of.
Re: IPv6 DNSBL/WL design, was Fwd: [Asrg] draft-levine-iprangepub-01
On 12/30/2010 9:49 PM, John R Levine wrote: I'm not wedded to the CNAME hack. Actually, I was thinking about that. Consider a hack on a DNS server that gives all records an absolute expiry time that marches forward in (say) 5-minute intervals. Then when the DNS server is queried, the TTL is computed to be the difference between the current time and the next absolute expiry. That had occurred to me. Another possibility is to embed serial numbers in the records and if the client sees it's out of sync, it goes back to the root and starts over. PS: While you're at it, SMTP needs to be replaced, too. Apples and oranges. SMTP was designed for sending email, which it excels at. The DNS was designed as essentially a distributed lookup table. It was never designed to be warped into a read-only B-tree. :) Snerk. SMTP was designed for a network with no security where everyone behaved themselves and all the mail was ASCII text. We shoehorned in formatted mail and file attachments with MIME, kludged in some security with S/MIME and later SPF and DKIM, and are now in the midst of a really, really big kludge to try to add Unicode addressing in EAI. It passed its best-by date decades ago, but it shares with the DNS the fact that it exists, and the putatively better alternatives don't.* Haw. All that a SSL certificate in S/MIME does is verify that the e-mail sender is who they say that they are. But, they can still screw you over. Nothing is stopping someone from putting up a SSL website that has non-working john thomas enlargement pills on it, obtaining a SSL certificate from Verisign, and proceeding to screw the public out of their hard-earned money. Verisign will happily give them a SSL certificate because they AREN'T HIDING. Nothing is preventing a spammer from doing the same with s/MIME or SPF or DKIM or any of the protocols you think kludged in security Well they didn't. I get plenty of spam that passes SPF and DKIM. Even if SMTP made SSL certificates mandatory so that you knew EXACTLY who was spamming you, they WOULD STILL SPAM you. S/MIME does absolutely nothing to guarentee that the data you get is legitimate. And this is why all of the SMTP alternatives that have been proposed over the years have FAILED. It is not because of entrenchment - entrenchment didn't help to limit hard drive sizes so that even modern hard drives of 2TB will work on old motherboards that only speak CHS. It is because the idea that a new SMTP protocol will somehow guarantee that mail you get isn't full of garbage is a mirage. Why don't you ask all those people who invested in Bernie Madoff if knowing who screwed them over is going to help them get the money that Bernie made off with back? The UNIX philosophy has been to build more complex systems out of simple systems. It has survived to this day against ALL OTHER computing philosophies because of the fundamental correctness of this philosophy. And SMTP is the same philosophy. Unicode addressing should rightly be an add-on to a simpler system. And frankly the biggest proponent of EAI is China - and why do you think that this is? It's because the Chinese government wants to make it even more difficult for their citizens to interact with the rest of the world - they want to control information. Look at the Japanese who use the same characters and they don't have a problem with Latin-character e-mail addresses. China does - because they want to make it difficult for outsiders to send e-mail to their citizens. Their people have no problem sending mail to each other using Latin e-mail addresses because they can type those characters on their computers. But the rest of the world does not have Chinese characters on their keyboards so EAI makes it real hard to e-mail Chinese people - and that's the way that the Chinese government wants it. You don't see the same interest in EAI in Taiwan. Look at the intro attempts of diacritics into .cz. The Czechs themselves are opposed to it - they just yet again went against it in the 4th survey that was just taken last month. You want to know what their biggest objection is? More complicated access for foreigners, that's what it is. Read the story of the Tower of Babel. The general public knows it and does not want the Internet to turn into another Tower. End users all over most of the world WANT to interact with foreigners. They DO NOT want to have the Internet on their piece of the the world to become incompatible with everyone else. ASCII and the Latin characters that make it up are the lowest common denominator and everyone's whore and the end users of the world want it that way for e-mail addresses. Ted Regards, John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY I dropped the toothpaste, said Tom, crestfallenly. * - Well, OK, X.400 exists. Sort of.
Fwd: [Asrg] draft-levine-iprangepub-01
Hi all, I'm not sure whether that would be more appropriate for the dev list, but I guess this is relevant/of interest to the SpamAssassin project, and I don't know whether this has caught attention here yet. John in his draft mentioned below is very right to point out that simply applying the IPv4-DNSxL paradigm to IPv6 has the potential to drain resources from DNS caches. Especially, a spammer may cause DNS caches to fill with useless data by enumerating /128s in a /64 interface, and thus causing a spam filter to start DNS queries for all those /128s. I'm not yet convinced that John's proposal of a B-tree search encoded in DNS TXT records is really a good choice (it adds a lot of complexity to a previously extremely simple protocol), but maybe it's the best approach given the requirements. It makes most sense to discuss the proposal on the ASRG list. However, SA will be a major user of this protocol, so likely some implementation ideas can be discussed here as well. Personally, I'm also open for other ideas or approaches that make it feasible to query reputation lists in an IPv6 world, be it over DNS or some other transport protocol. -- Matthias -- Forwarded message -- From: John R. Levine jo...@iecc.com Date: Wed, Dec 29, 2010 at 5:48 PM Subject: [Asrg] draft-levine-iprangepub-01 To: Anti Spam Research Group a...@irtf.org I've done another version of it, with mostly editorial changes. Comments as always welcome. I guess now I should implement it. http://tools.ietf.org/html/draft-levine-iprangepub-01 Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly ___ Asrg mailing list a...@irtf.org http://www.irtf.org/mailman/listinfo/asrg
Re: [Asrg] draft-levine-iprangepub-01
On Wed, 29 Dec 2010 21:09:42 +0100 Matthias Leisi matth...@leisi.net wrote: I'm not sure whether that would be more appropriate for the dev list, but I guess this is relevant/of interest to the SpamAssassin project, and I don't know whether this has caught attention here yet. In the draft, John asserts: For blacklists, an obvious approach would be to limit the granularity of DNSBLs, so that, say, each /64 had a separate listing, and the queries only used the high 64 bits of each address. While this might limit the damage from DNSBL queries, it is not helpful for DNS whitelists, which by their nature list individual IP addresses I'm not sure I agree with that. The smallest unit of IPv6 address space allocated by a provider (even to an end-user) is likely to be a /64, so I don't see why whitelists can't list /64's too. Essentially, I disagree with the phrase which by their nature list individual IP addresses. Regards, DAvid.
Re: [Asrg] draft-levine-iprangepub-01
On Wed, 29 Dec 2010 15:26:07 -0500, David F. Skoll d...@roaringpenguin.com wrote: On Wed, 29 Dec 2010 21:09:42 +0100 Matthias Leisi matth...@leisi.net wrote: I'm not sure whether that would be more appropriate for the dev list, but I guess this is relevant/of interest to the SpamAssassin project, and I don't know whether this has caught attention here yet. In the draft, John asserts: For blacklists, an obvious approach would be to limit the granularity of DNSBLs, so that, say, each /64 had a separate listing, and the queries only used the high 64 bits of each address. While this might limit the damage from DNSBL queries, it is not helpful for DNS whitelists, which by their nature list individual IP addresses I'm not sure I agree with that. The smallest unit of IPv6 address space allocated by a provider (even to an end-user) is likely to be a /64, so I don't see why whitelists can't list /64's too. Essentially, I disagree with the phrase which by their nature list individual IP addresses. Regards, DAvid. I'd wonder at the DNS traffic, I may be wrong but this looks like between 4 and 24 look-ups per check. DoS? Nigel
Re: [Asrg] draft-levine-iprangepub-01
On Wed, Dec 29, 2010 at 9:26 PM, David F. Skoll d...@roaringpenguin.com wrote: I'm not sure I agree with that. The smallest unit of IPv6 address space allocated by a provider (even to an end-user) is likely to be a /64, so I don't see why whitelists can't list /64's too. Essentially, I disagree with the phrase which by their nature list individual IP addresses. It's not certain that ISPs will always allocate /64. Some may allocate /56 or something entirely different, and shared hosting providers may allocate smaller ranges to their customers (why not an individual IP to each customer?). Enterprise users may be happy with announcing specific /128s for their one to four mailservers. And so on: Regardless of allocation policy, a protocol must support varying netmask lengths. Specifying /64 only or /128 only is not going to work. For dnswl.org, I see situations where we will use an ISP-provided-to-an-enduser range (/64 or whatever), and others where we will have smaller ranges (down to /128s, and possibly something in between /64 and /128). -- Matthias
Re: [Asrg] draft-levine-iprangepub-01
On Wed, 29 Dec 2010 21:34:47 +0100 Matthias Leisi matth...@leisi.net wrote: It's not certain that ISPs will always allocate /64. Some may allocate /56 or something entirely different, Bigger than /64 is no problem. and shared hosting providers may allocate smaller ranges to their customers (why not an individual IP to each customer?). Because then your routing table gets insane. And so on: Regardless of allocation policy, a protocol must support varying netmask lengths. Specifying /64 only or /128 only is not going to work. Limiting the granularity of a whitelist to a /64 seems pretty reasonable to me. And if you're on a network where some hosts in the /64 are good and some are bad... then tough luck; you don't get whitelisted. Pick a provider with a sane allocation policy. :) For dnswl.org, I see situations where we will use an ISP-provided-to-an-enduser range (/64 or whatever), and others where we will have smaller ranges (down to /128s, and possibly something in between /64 and /128). If dnswl.org and others announced that (1) they would whitelist only to the granularity of a /64 and (2) any providers that put different customers in the same /64 would be ineligible for whitelisting, economics would quickly move providers to allocating at least a /64 to each customer. http://tools.ietf.org/html/rfc3177 allows for assignment of a /128, but only under quite restricted circumstances. See 3. Address Delegation Recommendations in that RFC. (Yes, it's only informational, but it should still carry a fair amount of weight.) Regards, David.
Re: [Asrg] draft-levine-iprangepub-01
On Wed, Dec 29, 2010 at 9:52 PM, David F. Skoll d...@roaringpenguin.com wrote: and shared hosting providers may allocate smaller ranges to their customers (why not an individual IP to each customer?). Because then your routing table gets insane. They may allocate the IPs in a virtualisation layer. If dnswl.org and others announced that (1) they would whitelist only to the granularity of a /64 and (2) any providers that put different customers in the same /64 would be ineligible for whitelisting, economics would quickly move providers to allocating at least a /64 to each customer. Today, querying IPv4 DNSxLs is more or less limited to individual IPs. Making a new protocol that has more flexibility is very much needed - one size will not fit all, especially not in the protocol design. http://tools.ietf.org/html/rfc3177 allows for assignment of a /128, but only under quite restricted circumstances. See 3. Address Delegation Recommendations in that RFC. (Yes, it's only informational, but it should still carry a fair amount of weight.) And it argues to assign /48s to end-user systems, which does not seem to be a very well established practice today. -- Matthias
Re: [Asrg] draft-levine-iprangepub-01
On Wed, 29 Dec 2010 22:05:16 +0100 Matthias Leisi matth...@leisi.net wrote: Today, querying IPv4 DNSxLs is more or less limited to individual IPs. Making a new protocol that has more flexibility is very much needed - one size will not fit all, especially not in the protocol design. OK. But I think the draft is very complex and makes many DNS queries. Why not something like: Look up the first 64 bits: 0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.whitelist.example.org If you get back nothing, it's not whitelisted. If you get back 127.0.0.1, it's whitelisted. If you get back some magical value like 127.224.X.1, then you need to do a query against a /X (where X must be multiple of 4 and 64 X = 128). So if you wanted to list to the granularity of a /128, the query above would return 127.224.128.1 and you'd redo the query with the full 32-nibble address. This is less flexible, but results in only one query in the common case or two in the worst-case. (Naturally, you could use TXT records instead of magic A records; this is just for illustration.) List managers would have to be *very* careful not to list networks to a fine granularity unless they know for sure the manager of the network block takes precautions against IP address spoofing (using ingress/egress filtering or similar.) Regards, David.
Re: Fwd: [Asrg] draft-levine-iprangepub-01
I think the biggest problem with his draft is the following: For blacklists, an obvious approach would be to limit the granularity of DNSBLs, so that, say, each /64 had a separate listing, and the queries only used the high 64 bits of each address. While this might limit the damage from DNSBL queries, it is not helpful for DNS whitelists, which by their nature list individual IP addresses, and are likely to be far more popular with IPv6 mail than they have been with IPv4 mail. A host that is a legitimate host and thus deserving of being in a whitelist will not be enumerating every single IP address in a /64, one-per-message, the way that a spammer in a blacklist would be. Thus, we can safely make the assumption that any mailserver is going to follow the model of a single host per /64. Thus it will ALSO be just as useful for whitelists to have the same granularity - a /64 - as it would be for blacklists. What this really calls for is a reworking of the SpamAssassin code. SA is going to have to start caching the results of any IPv6 DNS BL queries for a set period of time, probably 2 days. Any IPv6 address in a BL needs to invalidate all other IPv6 addresses in the /64 that the IPv6 address is in for 2 days. There is no need to do further querying, nor is there a need for the scheme enumerated in the RFC draft. Ted On 12/29/2010 12:09 PM, Matthias Leisi wrote: Hi all, I'm not sure whether that would be more appropriate for the dev list, but I guess this is relevant/of interest to the SpamAssassin project, and I don't know whether this has caught attention here yet. John in his draft mentioned below is very right to point out that simply applying the IPv4-DNSxL paradigm to IPv6 has the potential to drain resources from DNS caches. Especially, a spammer may cause DNS caches to fill with useless data by enumerating /128s in a /64 interface, and thus causing a spam filter to start DNS queries for all those /128s. I'm not yet convinced that John's proposal of a B-tree search encoded in DNS TXT records is really a good choice (it adds a lot of complexity to a previously extremely simple protocol), but maybe it's the best approach given the requirements. It makes most sense to discuss the proposal on the ASRG list. However, SA will be a major user of this protocol, so likely some implementation ideas can be discussed here as well. Personally, I'm also open for other ideas or approaches that make it feasible to query reputation lists in an IPv6 world, be it over DNS or some other transport protocol. -- Matthias -- Forwarded message -- From: John R. Levinejo...@iecc.com Date: Wed, Dec 29, 2010 at 5:48 PM Subject: [Asrg] draft-levine-iprangepub-01 To: Anti Spam Research Groupa...@irtf.org I've done another version of it, with mostly editorial changes. Comments as always welcome. I guess now I should implement it. http://tools.ietf.org/html/draft-levine-iprangepub-01 Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly ___ Asrg mailing list a...@irtf.org http://www.irtf.org/mailman/listinfo/asrg
