Re: Repeated Blacklisting / IP reputation
On Tue, Sep 15, 2009 at 09:22:02PM -0400, Christopher Morrow wrote: build expertise on managing it. If you go to SpamHaus you will see a major ISP and their netblocks listed and associated with known spammers. What is this ISP doing about this? Nothing! ?My guess is that they look at their 'nothing' that you can see? or nothing? or something you can't see or that's taking longer than you'd expect/like? There certainly are bad actors out there, but I think the majority are doing things to keep clean, perhaps not in the manner you would like (or the speed you would like or with as much public information as you'd like). [ engage cynical mode] It's the responsibilty of all operations to ensure that they're not persistent or egregious sources of abuse. *Some* operations handle that reasonably well, but unfortunately many do not -- which is why there are now hundreds of blacklists (of varying intent, design, operation, and so on). If ISPs et.al. were doing their jobs properly, there would be no need for any of these to exist. But they're not, which is why so many people have taken the time and trouble to create them. Overall ISP performance in re abuse handling is miserable and has been for many years, and that includes everything from a lack of even perfunctory due diligence (30 seconds with Google) to failure to handle the abuse role address properly and promptly to alarming naivete' (what did you THINK they were doing with an entire /24 full of nonsense domain names?) to deployment of anti-spam measures that make the problem worse and inflict abuse on third parties to... This is hardly surprising: there are few, if any, consequences for doing so, and of course it's far more profitable to not just turn a blind eye to abuse (which used to be common) but moreso these days to actively assist in it with a smile and a wink and a hand extended for the payoff, while simultaneously making a public show of deep concern and issuing press releases that say We take the X problem seriously... and participating in working groups that studiously avoid the actual problems -- or better yet, which invite well-known/long-time abusers to have a seat at the table. ---Rsk
RE: Repeated Blacklisting / IP reputation, replaced by registered use
I think ARIN is no party to contact all RBL's and do any cleanup of 'contaminated' address space. The only steps ARIN might do are: - When requesting address space, one should be able to indicate whether receiving previous used address space would be unwanted or not. - When assigning address space, ARIN should notify receivers if it's re-used or virgin address space. - When address space got returned to ARIN and there is evidence of abuse, they have to mark that address space as 'contaminated' and only re-assign that space to new end-users who have indicated to have no problem with that. With kind regards, Michiel Klaver IT Professional
Re: Repeated Blacklisting / IP reputation
Well, I haven't even had coffee yet and... Get the removals: curl -ls http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html | grep Remove | grep -v PRE Get the additions: mahannig$ curl -ls http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html | grep Add | grep -v PRE I'm sure someone else could write something far more elegant, but elegance isn't always required. :-) Best, Marty On Mon, Sep 14, 2009 at 10:21 PM, Martin Hannigan mar...@theicelandguy.comwrote: On Mon, Sep 14, 2009 at 2:58 PM, Justin Shore jus...@justinshore.comwrote: Frank Bulk wrote: With scarcity of IPv4 addresses, organizations are more desperate than ever to receive an allocation. If anything, there's more of a disincentive than ever before for ARIN to spend time on netblock sanitization. I do think that ARIN should inform the new netblock owner if it was previously owned or not. But if ARIN tried to start cleaning up a netblock before releasing it, there would be no end to it. How could they check against the probably hundreds of thousands private blocklist? They could implement a process by which they announce to a mailing list of DNSBL providers that a given assignment has been returned to the RIR and that it should be cleansed from all DNSBLs. You mean like this? http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html -M -- Martin Hannigan mar...@theicelandguy.com p: +16178216079 Power, Network, and Costs Consulting for Iceland Datacenters and Occupants
RE: Repeated Blacklisting / IP reputation
I'd be more than happy to see this, with the added caveat that anyone that returned address space to ARIN that was subsequently marked as 'contaminated', should undergo a review process when attempting to obtain new address space. Charge them for the review process Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. Another option, is to hit them where it matters. Assign fines and fees for churning address space and returning it as contaminated. Set the fee's on a sliding scale based on the amount of contamination and churn. the more contamination, the higher the fee. Shawn Somers Michiel Klaver wrote: - Message: 3 Date: Tue, 15 Sep 2009 11:57:58 +0200 From: Michiel Klaver mich...@klaver.it Subject: RE: Repeated Blacklisting / IP reputation, replaced by registered use To: Azinger, Marla marla.azin...@frontiercorp.com, John Curran jcur...@arin.net, nanog@nanog.org nanog@nanog.org Message-ID: 4aaf6526.9000...@klaver.it Content-Type: text/plain; charset=UTF-8; format=flowed I think ARIN is no party to contact all RBL's and do any cleanup of 'contaminated' address space. The only steps ARIN might do are: - When requesting address space, one should be able to indicate whether receiving previous used address space would be unwanted or not. - When assigning address space, ARIN should notify receivers if it's re-used or virgin address space. - When address space got returned to ARIN and there is evidence of abuse, they have to mark that address space as 'contaminated' and only re-assign that space to new end-users who have indicated to have no problem with that. With kind regards, Michiel Klaver IT Professional
Re: Repeated Blacklisting / IP reputation
Martin Hannigan wrote: Well, I haven't even had coffee yet and... Get the removals: curl -ls http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html | grep Remove | grep -v PRE Get the additions: mahannig$ curl -ls http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html | grep Add | grep -v PRE That appears to be it. I've also been told that there is a RSS feed of the same thing. My understanding is that a posting is made to the mailing list or RSS feed when a new subnet is assigned. I'd like to see them do something with the assignment is first returned to ARIN, not months later when the assignment is ready to be handed out again. I think the extra time would help those people that download copies of the DNSBL zone files and manually import them once a week or less often. Lots of place still use the zone files. Personally I prefer to do so too, rather than tie my mail system reliability on an outside source that may or may not tell me when they have problems that affect my service. GoDaddy and their hosted mail service would be a great example since they can't be bothered to update their DNSBL zone files. Their mail admins are using a copy of SORBS that is 3 years old. 3 damn years old. How do I know this? 3 years ago a mistake in a Squid configuration turned one of my services into an open proxy for about a week. Even today mail from that server to a domain with mail hosted at GoDaddy results in a bounce citing the ancient SORBS listing as the reason. Thanks for the pointer. Looks like they've already thought of what I suggested and implemented a solution. I still voice for announcing returned assignment instead of announcing when an old assignment gets reassigned. Thanks Justin
RE: Repeated Blacklisting / IP reputation
The mailing sent daily contains both. -Original Message- From: Justin Shore [mailto:jus...@justinshore.com] Sent: Tuesday, September 15, 2009 11:18 AM To: Martin Hannigan Cc: NANOG list Subject: Re: Repeated Blacklisting / IP reputation Martin Hannigan wrote: Well, I haven't even had coffee yet and... Get the removals: curl -ls http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html | grep Remove | grep -v PRE Get the additions: mahannig$ curl -ls http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html | grep Add | grep -v PRE That appears to be it. I've also been told that there is a RSS feed of the same thing. My understanding is that a posting is made to the mailing list or RSS feed when a new subnet is assigned. I'd like to see them do something with the assignment is first returned to ARIN, not months later when the assignment is ready to be handed out again. I think the extra time would help those people that download copies of the DNSBL zone files and manually import them once a week or less often. Lots of place still use the zone files. Personally I prefer to do so too, rather than tie my mail system reliability on an outside source that may or may not tell me when they have problems that affect my service. GoDaddy and their hosted mail service would be a great example since they can't be bothered to update their DNSBL zone files. Their mail admins are using a copy of SORBS that is 3 years old. 3 damn years old. How do I know this? 3 years ago a mistake in a Squid configuration turned one of my services into an open proxy for about a week. Even today mail from that server to a domain with mail hosted at GoDaddy results in a bounce citing the ancient SORBS listing as the reason. Thanks for the pointer. Looks like they've already thought of what I suggested and implemented a solution. I still voice for announcing returned assignment instead of announcing when an old assignment gets reassigned. Thanks Justin
Re: Repeated Blacklisting / IP reputation
On Tue, 15 Sep 2009 08:01:48 PDT, Shawn Somers said: Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. You *do* realize that the people you're directing that paragraph at are able to say with a totally straight face: We're doing nothing wrong and we have *no* idea why we end up in so many local block lists? pgpL8Pxlc5CTN.pgp Description: PGP signature
Re: Repeated Blacklisting / IP reputation
so... this thread has a couple of really interesting characteristics. a couple are worth mentioning more directly (they have been alluded to elsewhere)... Who gets to define bad - other than a blacklist operator? Are the common, consistent defintions of contamination? If these are social/political - recognise that while the ARIN region is fairly consistent in its general use and interpretation of law, there are known varients - based on soveriegn region. this whole debate/discussion seems based on the premise that there are well known, consistent, legally defendable choices for defining offensive behaviours. and pretty much all of history shows us this is not the case. (is or is not a mother nursing her child in public pornographic?) So - I suspect that in the end, a registry (ARIN) or an ISP (COMCAST) is only going to be able to tell you a few things about the prefix you have been handed. a) its virginal - never been used (that we know of) b) its been used once. c) it has a checkered past and it will be up to the receipient to trust/accept the resource for what it currently is or chose to reject it and find soliace elsewhere. --bill On Tue, Sep 15, 2009 at 04:31:04PM -0400, Christopher Morrow wrote: On Tue, Sep 15, 2009 at 4:23 PM, valdis.kletni...@vt.edu wrote: On Tue, 15 Sep 2009 08:01:48 PDT, Shawn Somers said: Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. You *do* realize that the people you're directing that paragraph at are able to say with a totally straight face: We're doing nothing wrong and we have *no* idea why we end up in so many local block lists? Also, you can very well disable new allocations to Spammer-Bob, did you also know his friend Sue is asking now for space? Sue is very nice, she even has cookies... oh damn after we allocated to her we found out she's spamming :( Spammers have a lot of variables to change in this equation, RIR's dont always have the ability to see all of the variables, nor correlate all of the changes they see :( -Chris
Re: Repeated Blacklisting / IP reputation
I believe there is another side to that argument as well. If I operate a regional ISP and request address space for dynamic address pools I am aware of a few things: 1) I am fully aware that there is a chance a customer's system could become infected and generate millions of malicious messages/packets/ traffic. 2) I am also aware that it is possible that that one machine could have any number of IP addresses during the course of the week; therefore, it would be possible that they could 'contaminate' an entire /24 3) I know that if I'm made aware of the zombified machine that I'll disable access to the customer quickly; however, the damage has usually already been done. 4) Do I actually care if one of my dynamic address blocks are in a DNSBL? Not at all. They should be using my mail server anyways. Should I have to go through and make sure that every single IP address/block is 'clean' before returning the allocation to ARIN? I can say with utmost confidence I don't care because I no longer need them. If my ability to receive new allocations required that I clean up a dynamic address block before receiving a new one I would take better care of my blocks; however, it may be cheaper just to keep the old block (null route it) and ask for another one. The question becomes: Where do you draw the 'contamination' line? A network may be using a block well within what we would consider 'reasonable' usage; however, the block may become 'unusable' for certain purposes. Should they too be denied further address space? If thats the case every broadband provider out there should be cut off because they're customers keep getting infected and are used to DDOS/ SPAM/Exploit our networks. What I'm trying to say in a long-winded and round about way is simple --- The contamination doesn't always happen 'on purpose' or with any foresight and it may not be an entire block that is bad. Everyone is guilty at some point of having a few 'dirty' IPs on their network... and I'm sure all of us have left many dirty because god only knows where all it is blocked. On Sep 15, 2009, at 4:23 PM, valdis.kletni...@vt.edu wrote: On Tue, 15 Sep 2009 08:01:48 PDT, Shawn Somers said: Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. You *do* realize that the people you're directing that paragraph at are able to say with a totally straight face: We're doing nothing wrong and we have *no* idea why we end up in so many local block lists?
on naming conventions (was: Re: Repeated Blacklisting / IP reputation)
on Tue, Sep 08, 2009 at 09:57:58AM -0500, Tom Pipes wrote: [...] We have done our best to ensure these blocks conform to RFC standards, including the proper use of reverse DNS pointers. Sorry to jump in so late, been catching up from vacation. I'm checking out the PTRs for the /18 you mention, and I see that you've used a few different naming conventions, some of which are friendly to those who block on dot-separated substrings, some of which are confusing, and some of which are custom to specific clients. If I could speak on behalf of the tens of thousands of mail admins out there for a minute, I'd ask that instead of (e.g.) 69.197.115.62: 69-197-115-62-dynamic.t6b.com you instead use a dot to separate the 'dynamic' from the generated IP-based hostname part, a la 69.197.115.62: 69-197-115-62.dynamic.t6b.com This allows admins of most FOSS MTAs to simply deny traffic from all of those hosts on the grounds that they are dynamically assigned, for example in sendmail's access.db: Connect:dynamic.t6b.com ERROR:5.7.1:550 Go away, dynamic user. If you choose not to, it doesn't bother me; I've got a rather extensive set of regular expressions that can handle those naming conventions, but the rest of the mail admins may find it more friendly were you to do so. Additionally, it may also be useful to indicate what sort of access is being provided, so for dialups you might want to do 69.197.115.62: 69-197-115-62.dialup.dynamic.t6b.com (Note: not 'dynamic.dialup.t6b.com', most people care more about whether a host is dynamic at least in the context of antispam operations). I also note that the vast majority of the /18 simply lacks PTRs at all; you also mix statics and dynamics (though on different /24s, eg 69.197.106, 69.197.107, 69.197.108 seem static where 69.197.110, 69.197.111, and 69.197.115 do not, with more statics seen in 69.197.117 and 69.197.118 ff.) and don't seem to SWIP the statics or indicate in whois which are dynamic pools. All of these are likely to result in unfunny errors by DNSBL operators if they decide that you're serious and the whole /18 is dynamic based on a preponderance of hosts in some /24s with dynamic-appearing names AND a lack of evidence otherwise in the whois record. Of course, if you follow MAAWG's port 25 blocking BCP, it's moot as far as the dynamics go. Ultimately, you'd want to make sure any static customer intending to provide mail services have their own custom PTR(s) for those hosts, in their domains (not yours). HTH, Steve -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2553 w: http://hesketh.com/ antispam news and intelligence to help you stop spam: http://enemieslist.com/
Re: Repeated Blacklisting / IP reputation
On Tue, Sep 15, 2009 at 5:31 PM, Zaid Ali z...@zaidali.com wrote: I think costs of maintaining an abuse helpdesk is a big factor here. I don't see many ISP's putting money and resources into an abuse helpdesk and this is because it is low cost to obtain a Netblock so why should one employ and have you ever had to re-number a customer, several customers, a hundred?? 'getting a new netblock is low cost' is hardly an accurate statement, especially if you keep in mind that you have to justify the usage of old netblocks in order to obtain the new one. build expertise on managing it. If you go to SpamHaus you will see a major ISP and their netblocks listed and associated with known spammers. What is this ISP doing about this? Nothing! My guess is that they look at their 'nothing' that you can see? or nothing? or something you can't see or that's taking longer than you'd expect/like? There certainly are bad actors out there, but I think the majority are doing things to keep clean, perhaps not in the manner you would like (or the speed you would like or with as much public information as you'd like). From the outside most ISP operations look quite opaque, proclaiming 'Nothing is being done' simply looks uneducated and shortsighted. bottom $$ and look at Spamming customer A and say crap we will be spending $$$ on this customer just to get them off SpamHaus so just leave it, we are afterall in the bandwidth business. If ARIN were to say to this major ISP that they wont allocate more addresses to them until they adhere to an AUP then maybe the game will change but the bigger question here is should ARIN get into this kind of policy. doubtful that: 1) arin would say this (not want to be net police), 2) isp's couldn't show (for the vast majority of isps) that they are in fact upholding their AUP. -chris On Sep 15, 2009, at 1:31 PM, Christopher Morrow wrote: On Tue, Sep 15, 2009 at 4:23 PM, valdis.kletni...@vt.edu wrote: On Tue, 15 Sep 2009 08:01:48 PDT, Shawn Somers said: Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. You *do* realize that the people you're directing that paragraph at are able to say with a totally straight face: We're doing nothing wrong and we have *no* idea why we end up in so many local block lists? Also, you can very well disable new allocations to Spammer-Bob, did you also know his friend Sue is asking now for space? Sue is very nice, she even has cookies... oh damn after we allocated to her we found out she's spamming :( Spammers have a lot of variables to change in this equation, RIR's dont always have the ability to see all of the variables, nor correlate all of the changes they see :( -Chris
Re: Repeated Blacklisting / IP reputation
On Tue, Sep 15, 2009 at 4:46 PM, bmann...@vacation.karoshi.com wrote: so... this thread has a couple of really interesting characteristics. a couple are worth mentioning more directly (they have been alluded to elsewhere)... as always, despite your choice in floral patterned shirts :) good comments/questions. Who gets to define bad - other than a blacklist operator? Are the common, consistent defintions of contamination? nope, each BL (as near as I can tell) has their own criteria (with some overlaps to be certain) and they all have their own set of rules that they either break at-will or change when it suits them. Their incentives are not aligned with actually getting the problem resolved, sadly... and they really don't have any power to resolve problems anyway. If these are social/political - recognise that while the ARIN region is fairly consistent in its general use and interpretation of law, there are known varients - based on soveriegn region. Yup, you don't like my business how about I move to the caymans where it's no longer illegal? :( The Internet brings with it some interesting judicial/jurisdictional baggage. this whole debate/discussion seems based on the premise that there are well known, consistent, legally defendable choices for defining offensive behaviours. and pretty much all of history shows us this is not the case. There are really two discussions, I think somewhere along the path they were conflated: 1) newly allocated from IANA netblocks show up to end customers and reachability problems ensue. (route-filters and/or firewall filters) 2) newly re-allocated netblocks show up with RBL baggage (rbls and smtp blocks at the application layer) For #1 there was some work (rbush and prior to that Jon Lewis 69block.org?) showing that folks 'never' alter their 'bogon route filters' or 'bogon access-list entries'. For #2 ARIN may have a solution in place, if it were more publicly known (rss feed of allocations, care of RS and marty hannigan pointers) that RBL operators could use to clean out entries in their lists providing a better service to their 'users' even, perish the thought! (is or is not a mother nursing her child in public pornographic?) or SI Swinsuit edition depending on the part of the world you are in, yes, or even YouTube videos, weee! So - I suspect that in the end, a registry (ARIN) or an ISP (COMCAST) is only going to be able to tell you a few things about the prefix you have been handed. a) its virginal - never been used (that we know of) b) its been used once. c) it has a checkered past I actually don't think it's a help for ARIN to say anything here, since they can never know all the RBL's and history for a netblock, and they can't help in the virginal case since they don't run network-wide filters. A FAQ that says some of the above with some pointers to testing harnesses to use may be useful. Some tools for network operators to use in updating things in a timely fashion may be useful. Better/wider/louder notification 'services' for new block allocations from IANA - RIR's may be useful. Not everyone who runs a router reads their local 'nog' list... Leo Vegoda does a great job tell us about RIPE allocations, Someone does the same for ARIN (drc maybe??) and I'm not certain I recall who's last announced APNIC block yahtzee. Where else is this data available? In a form that your avg enterprise network op may notice? and it will be up to the receipient to trust/accept the resource for what it currently is or chose to reject it and find soliace elsewhere. 'solace elsewhere'... dude there is no 'elsewhere'. -Chris (and yes, I'm yanking your chain about the shirts...) --bill On Tue, Sep 15, 2009 at 04:31:04PM -0400, Christopher Morrow wrote: On Tue, Sep 15, 2009 at 4:23 PM, valdis.kletni...@vt.edu wrote: On Tue, 15 Sep 2009 08:01:48 PDT, Shawn Somers said: Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. You *do* realize that the people you're directing that paragraph at are able to say with a totally straight face: We're doing nothing wrong and we have *no* idea why we end up in so many local block lists? Also, you can very well disable new allocations to Spammer-Bob, did you also know his friend Sue is asking now for space? Sue is very nice, she even has cookies... oh damn after we allocated to her we found out she's spamming :( Spammers have a lot of variables to change in this equation, RIR's dont always have the ability to see all of the variables, nor correlate all of the changes they see :( -Chris
Re: Repeated Blacklisting / IP reputation
On Tue, Sep 15, 2009 at 09:34:14PM -0400, Christopher Morrow wrote: On Tue, Sep 15, 2009 at 4:46 PM, bmann...@vacation.karoshi.com wrote: so... this thread has a couple of really interesting characteristics. a couple are worth mentioning more directly (they have been alluded to elsewhere)... as always, despite your choice in floral patterned shirts :) good comments/questions. humph... at least I wear pants. Who gets to define bad - other than a blacklist operator? Are the common, consistent defintions of contamination? nope, each BL (as near as I can tell) has their own criteria (with trick question... each ISP gets to define good/bad on their own merits or can outsource it to third parties. 1) newly allocated from IANA netblocks show up to end customers and reachability problems ensue. (route-filters and/or firewall filters) 2) newly re-allocated netblocks show up with RBL baggage (rbls and smtp blocks at the application layer) you forgot #3 ... a clean IANA block that was borrowed for a while .. and already shows up in some filter lists. So - I suspect that in the end, a registry (ARIN) or an ISP (COMCAST) is only going to be able to tell you a few things about the prefix you have been handed. a) its virginal - never been used (that we know of) b) its been used once. c) it has a checkered past I actually don't think it's a help for ARIN to say anything here, since they can never know all the RBL's and history for a netblock, and they can't help in the virginal case since they don't run network-wide filters. not RBL specific ... a) this block came directly from IANA and has never been previously allocated in/through the IANA/RIR process b) this block has had one registered steward in recorded history c) this block has been in/out of the RIR/registry system more than once. A FAQ that says some of the above with some pointers to testing harnesses to use may be useful. Some tools for network operators to use in updating things in a timely fashion may be useful. Better/wider/louder notification 'services' for new block allocations from IANA - RIR's may be useful. indeed - I'd like to see the suite extended to the ISPs as well, esp if such tricks will be used in v6land... last announced APNIC block yahtzee. Where else is this data available? In a form that your avg enterprise network op may notice? oh... I'd suggest some of the security lists might be a good channel. and it will be up to the receipient to trust/accept the resource for what it currently is or chose to reject it and find soliace elsewhere. 'solace elsewhere'... dude there is no 'elsewhere'. and yet... Jimmy and Warren Buffet will tell you its always 1700 somewhere and if that doesn't work, whip out the NAT and reuse 10.0.0.0 -again- :) -Chris (and yes, I'm yanking your chain about the shirts...) --bill On Tue, Sep 15, 2009 at 04:31:04PM -0400, Christopher Morrow wrote: On Tue, Sep 15, 2009 at 4:23 PM, valdis.kletni...@vt.edu wrote: On Tue, 15 Sep 2009 08:01:48 PDT, Shawn Somers said: Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. You *do* realize that the people you're directing that paragraph at are able to say with a totally straight face: We're doing nothing wrong and we have *no* idea why we end up in so many local block lists? Also, you can very well disable new allocations to Spammer-Bob, did you also know his friend Sue is asking now for space? Sue is very nice, she even has cookies... oh damn after we allocated to her we found out she's spamming :( Spammers have a lot of variables to change in this equation, RIR's dont always have the ability to see all of the variables, nor correlate all of the changes they see :( -Chris
Re: Repeated Blacklisting / IP reputation
On Tue, Sep 15, 2009 at 10:29 PM, bmann...@vacation.karoshi.com wrote: On Tue, Sep 15, 2009 at 09:34:14PM -0400, Christopher Morrow wrote: On Tue, Sep 15, 2009 at 4:46 PM, bmann...@vacation.karoshi.com wrote: so... this thread has a couple of really interesting characteristics. a couple are worth mentioning more directly (they have been alluded to elsewhere)... as always, despite your choice in floral patterned shirts :) good comments/questions. humph... at least I wear pants. you have something against skirts? or dresses? always with the pants with you!! shakey fist Who gets to define bad - other than a blacklist operator? Are the common, consistent defintions of contamination? nope, each BL (as near as I can tell) has their own criteria (with trick question... each ISP gets to define good/bad on their own merits or can outsource it to third parties. sure... outsourcing in this case often happens without a real business relationship. 1) newly allocated from IANA netblocks show up to end customers and reachability problems ensue. (route-filters and/or firewall filters) 2) newly re-allocated netblocks show up with RBL baggage (rbls and smtp blocks at the application layer) you forgot #3 ... a clean IANA block that was borrowed for a while .. and already shows up in some filter lists. ok... but we can't ever really know that Verizon uses 114/8 and 104/8 internally can we? (and has/may leak this to external parties on occasion by mistake) So - I suspect that in the end, a registry (ARIN) or an ISP (COMCAST) is only going to be able to tell you a few things about the prefix you have been handed. a) its virginal - never been used (that we know of) b) its been used once. c) it has a checkered past I actually don't think it's a help for ARIN to say anything here, since they can never know all the RBL's and history for a netblock, and they can't help in the virginal case since they don't run network-wide filters. not RBL specific ... a) this block came directly from IANA and has never been previously allocated in/through the IANA/RIR process b) this block has had one registered steward in recorded history c) this block has been in/out of the RIR/registry system more than once. Ok, is this in the final email from hostmaster@ to 'enduser@'? or somewhere else? what's the recourse when someone says: But I don't want a USED netblock, it my have the herp! I'm trying to see if ARIN can say something of use here without raising its costs or causing extra/more confusion to the end-site(s). A FAQ that says some of the above with some pointers to testing harnesses to use may be useful. Some tools for network operators to use in updating things in a timely fashion may be useful. Better/wider/louder notification 'services' for new block allocations from IANA - RIR's may be useful. indeed - I'd like to see the suite extended to the ISPs as well, esp if such tricks will be used in v6land... last announced APNIC block yahtzee. Where else is this data available? In a form that your avg enterprise network op may notice? oh... I'd suggest some of the security lists might be a good channel. sure, most of those folks also read nanog-l, this won't also reach enterprise folk... (admittedly it's hard to reach 'everyone', but spammers seem to be able to...) and it will be up to the receipient to trust/accept the resource for what it currently is or chose to reject it and find soliace elsewhere. 'solace elsewhere'... dude there is no 'elsewhere'. and yet... Jimmy and Warren Buffet will tell you its always 1700 somewhere and if that doesn't work, whip out the NAT and reuse 10.0.0.0 -again- :) ha... :( -chris -Chris (and yes, I'm yanking your chain about the shirts...) --bill On Tue, Sep 15, 2009 at 04:31:04PM -0400, Christopher Morrow wrote: On Tue, Sep 15, 2009 at 4:23 PM, valdis.kletni...@vt.edu wrote: On Tue, 15 Sep 2009 08:01:48 PDT, Shawn Somers said: Anyone that intentionally uses address space in a manner that they know will cause it to become contaminated should be denied on any further address space requests. You *do* realize that the people you're directing that paragraph at are able to say with a totally straight face: We're doing nothing wrong and we have *no* idea why we end up in so many local block lists? Also, you can very well disable new allocations to Spammer-Bob, did you also know his friend Sue is asking now for space? Sue is very nice, she even has cookies... oh damn after we allocated to her we found out she's spamming :( Spammers have a lot of variables to change in this equation, RIR's dont always have the ability to see all of the variables, nor correlate all of the changes they
Re: Repeated Blacklisting / IP reputation
Christopher Morrow wrote: Spammers have a lot of variables to change in this equation, RIR's dont always have the ability to see all of the variables, nor correlate all of the changes they see :( Being a crimnal enterprise there are some tools in your kit that a legitimate business does not have. The problems becomes, how the raising the legitimacy bar more effectively discriminates against legitimate entities then crimnal one's. If a discriminatory measure were for example to raise the bar for new entrants that, by it's nature represents an Internet scale tragedy. joel -Chris
Re: Repeated Blacklisting / IP reputation
On Wed, Sep 16, 2009 at 12:08 AM, Joel Jaeggli joe...@bogus.com wrote: Christopher Morrow wrote: Spammers have a lot of variables to change in this equation, RIR's dont always have the ability to see all of the variables, nor correlate all of the changes they see :( Being a crimnal enterprise there are some tools in your kit that a legitimate business does not have. The problems becomes, how the that was my point, yes. raising the legitimacy bar more effectively discriminates against legitimate entities then crimnal one's. If a discriminatory measure were for example to raise the bar for new entrants that, by it's nature represents an Internet scale tragedy. I think we are in agreement on this issue, and the above actually. -Chris
Re: Repeated Blacklisting / IP reputation
On 9 Sep 2009, at 06:04, Peter Beckman wrote: How about a trial period from ARIN? You get your IP block, and you get 30 days to determine if it is clean or not. The reuse issue is possibly decades away in v6 land. The reuse issue can't really be solved for v4 in a year or two. Sounds like a waste of time to develop this idea further IMO. A
Re: Repeated Blacklisting / IP reputation
On Sun, Sep 13, 2009 at 12:45:03PM -0400, Christopher Morrow wrote: On Wed, Sep 9, 2009 at 11:48 PM, Mark Andrews ma...@isc.org wrote: skip a note about isc having quite a few legacy blocks Note we all could start using IPv6 and avoid this problem altogether. There is nothing stopping us using IPv6 especially for MTA's. that'd solve the spam problem... for a while at least. (no ipv6 traffic == no spam) 30% of our incoming IPv6 SMTP connections are spam. -- Tim
Re: Repeated Blacklisting / IP reputation
On Tue, Sep 08, 2009 at 11:44:44AM -0700, Wayne E. Bouchard wrote: Best practices for the public or subscription RBLs should be to place a TTL on the entry of no more than, say, 90 days or thereabouts. But there's no reason to do so, and a number of reasons not to, including the very high probabilityXcertainty that spammers would use this to rotate through multiple allocations at 91-day intervals. Best practice is to identify blocks that are owned (or effectively owned) by spammers and blacklist them until a need arises *on the receiving side* to remove those blocks. Yes, this is unfortunate, and draconian, and any number of other things, but the ISPs responsible for this situation should probably have considered this inevitable result before they decided to host well-known spammers that 60 seconds of due diligence would have identified, and subsequently to turn a blind eye to the abuse emanating from their networks. For example: Ron Guilmette has recently pointed out that notorious spammer Scott Richter has apparently hijacked *another* /16 block -- 150.230.0.0/16. I've dropped that block into various local blacklists, and in some cases, various local firewalls. The entry is essentially permanent, because there's no reason for me to make it otherwise. Perhaps one day ARIN will yank it back, along with all his other blocks, and blacklist him for life; but (a) I doubt it and (b) I'm not willing to wait. The best course of action for me is to just consider it scorched earth and move on. ---Rsk
Re: Repeated Blacklisting / IP reputation
On Sun, Sep 13, 2009 at 7:43 AM, John Curran jcur...@arin.net wrote: On Sep 11, 2009, at 6:52 PM, Martin Hannigan wrote: I honestly don't think that it's up to them to create a set-aside either, hence my comment about behind the scenes activities. I appreciate you detailing that, but I honestly don't think it matters since as you mentioned you get accused of this all of the time. I would expect that ICANN would not only follow the rules, but safeguard them as well. [ clip ] what would normally have been a behind the scenes implementation issue has now been publicly detailed, and I, for one, thank the IANA for their clear and timely communications on this matter. I do as well. ICANN does good work in this area and I would not want to appear as though I am saying otherwise. Numbering policy usually goes to the members of each of the RIR communities, just as the IANA to RIR policy did. The algorithm itself is great. The set-aside is the problem. This is not formation of global Internet numbering policy, it's implementation of the existing policy regarding IANA to RIR /8 block assignments. Regardless, the global nature of the Internet means that we'll all deal with connectivity issues with these blocks once they're allocated. Any and all efforts that the networking community can take now to get these blocks cleaned up now would be most helpful. Well, ok then :-). I agree to disagree. Anything that affects the flow or quality of IPv4 address space is a policy issue in my mind, especially when a justification for an action is linked to a social issue. I know that it was said that ICANN didn't really mean it when they said that they created this action with developing economies in mind, at least not in the way that it is defined[1], but it's hard to say after the fact. Best Regards, Marty 1. http://en.wikipedia.org/wiki/Developing_economies
Re: Repeated Blacklisting / IP reputation, replaced by registered use
On 9/13/09 12:49 PM, joel jaeggli wrote: Frank Bulk wrote: [] If anything, there's more of a disincentive than ever before for ARIN to spend time on netblock sanitization. This whole thread seems to be about shifting (I.E. by externalizing) the costs of remediation. presumably the entities responsible for the poor reputation aren't likely to pay... So heck, why not ARIN? perhaps because it's absurd on the face of it? how much do my fees go up in order to indemnify ARIN against the cost of a possible future cleanup? how many more staff do they need? Do I have to buy prefix reputation insurance as contingent requirement for a new direct assignm Perhaps ICANN could require registries establish a clearing-house, where at no cost, those assigned a network would register their intent to initiate bulk traffic, such as email, from specific addresses. Such a use registry would make dealing with compromised systems more tractable. I do think that ARIN should inform the new netblock owner if it was previously owned or not. We've got high quality data extending back through a least 1997 on what prefixes have been advertised in the DFZ, and of course from the ip reputation standpoint it doesn't so much matter if something was assigned, but rather whether it was ever used. one assumes moreover that beyond a certain point in the not too distant future it all will have been previously assigned (owned is the wrong word). But if ARIN tried to start cleaning up a netblock before releasing it, there would be no end to it. How could they check against the probably hundreds of thousands private blocklist? Note that they can't insure routability either, though as a community we've gotten used to testing for stale bogon filters. The issues created by IPv4 space churn is likely to be dwarfed by eventual adoption of IPv6. Registering intent to initiate bulk traffic, such as with SMTP, could help consolidate the administration of filters, since abuse is often from addresses that network administrators did not intend. A clearing-house approach could reduce the costs of administering filters and better insure against unintentional impediments. This approach should also prove more responsive than depending upon filters embedded within various types of network equipment. By limiting registration to those controlling the network, this provides a low cost means to control use of address space without the need to impose expensive and problematic layer 7 filters that are better handled by the applications. The size of the registered use list is likely to be several orders of magnitude smaller than the typical block list. Exceptions to the use list will be even smaller still. This registry would also supplant the guesswork involved with divining meaning of reverse DNS labels. -Doug
RE: Repeated Blacklisting / IP reputation, replaced by registered use
-Original Message- From: Douglas Otis [mailto:do...@mail-abuse.org] Sent: Monday, September 14, 2009 1:41 PM To: joel jaeggli Cc: NANOG list Subject: Re: Repeated Blacklisting / IP reputation, replaced by registered use On 9/13/09 12:49 PM, joel jaeggli wrote: Frank Bulk wrote: [] If anything, there's more of a disincentive than ever before for ARIN to spend time on netblock sanitization. This whole thread seems to be about shifting (I.E. by externalizing) the costs of remediation. presumably the entities responsible for the poor reputation aren't likely to pay... So heck, why not ARIN? perhaps because it's absurd on the face of it? how much do my fees go up in order to indemnify ARIN against the cost of a possible future cleanup? how many more staff do they need? Do I have to buy prefix reputation insurance as contingent requirement for a new direct assignm Perhaps ICANN could require registries establish a clearing-house, where at no cost, those assigned a network would register their intent to initiate bulk traffic, such as email, from specific addresses. Such a use registry would make dealing with compromised systems more tractable. If they would just comply with RFC 3514, such a registry would be unnecessary. This registry would also supplant the guesswork involved with divining meaning of reverse DNS labels. We could standardize a string to be used in rDNS of dynamic pools, if you want. Lee
Re: Repeated Blacklisting / IP reputation, replaced by registered use
On Sep 14, 2009, at 10:40 AM, Douglas Otis wrote: Perhaps ICANN could require registries establish a clearing-house, where at no cost, those assigned a network would register their intent to initiate bulk traffic, such as email, from specific addresses. ICANN can't require the RIRs do anything outside of what is specifically mentioned in global addressing policies. If you think this would be valuable and that it would make sense as a global addressing policy, then you should propose it in the RIR policy forums, get consensus amongst the five RIRs and have them forward it to ICANN as a global policy. Regards, -drc
RE: Repeated Blacklisting / IP reputation, replaced by registered use
Another one that could be discussed at the ARIN policy bof. Also, Im forwarding this to the ARIN ppml for any further discussion. Cheers Marla -Original Message- From: David Conrad [mailto:d...@virtualized.org] Sent: Monday, September 14, 2009 11:44 AM To: Douglas Otis Cc: NANOG list Subject: Re: Repeated Blacklisting / IP reputation, replaced by registered use On Sep 14, 2009, at 10:40 AM, Douglas Otis wrote: Perhaps ICANN could require registries establish a clearing-house, where at no cost, those assigned a network would register their intent to initiate bulk traffic, such as email, from specific addresses. ICANN can't require the RIRs do anything outside of what is specifically mentioned in global addressing policies. If you think this would be valuable and that it would make sense as a global addressing policy, then you should propose it in the RIR policy forums, get consensus amongst the five RIRs and have them forward it to ICANN as a global policy. Regards, -drc
Re: Repeated Blacklisting / IP reputation
Frank Bulk wrote: With scarcity of IPv4 addresses, organizations are more desperate than ever to receive an allocation. If anything, there's more of a disincentive than ever before for ARIN to spend time on netblock sanitization. I do think that ARIN should inform the new netblock owner if it was previously owned or not. But if ARIN tried to start cleaning up a netblock before releasing it, there would be no end to it. How could they check against the probably hundreds of thousands private blocklist? They could implement a process by which they announce to a mailing list of DNSBL providers that a given assignment has been returned to the RIR and that it should be cleansed from all DNSBLs. At this point the RIR has done their due diligence for notifying the blacklist community of the change and the onus is on the DNSBL maintainers to update their records. Of course this does nothing to cleanse the assignment in the hundreds of thousands of MTAs around the world. However this could be a good reason to not blacklist locally (or indefinitely at least) and to instead rely on a DNSBL maintained by people responsible for wiping returned assignments from their records when RIRs give the word. I suppose the mailing list could even be expanded to include mailing list admins if need be so that they could also receive the info and wipe their own internal DNSBLs. The list should be an announcement-only list with only the RIRs being able to post to it in a common and defined format. The announcement should be made as soon as the assignment is returned to the RIR, allowing for the cool off period of time for personal blacklists to catch up to the official ones. I would think that would be a fairly simple process to implement. It's not fool-proof by any means but it's better than doing nothing. It's a thought. Justin
Re: Repeated Blacklisting / IP reputation
On Mon, Sep 14, 2009 at 2:58 PM, Justin Shore jus...@justinshore.comwrote: Frank Bulk wrote: With scarcity of IPv4 addresses, organizations are more desperate than ever to receive an allocation. If anything, there's more of a disincentive than ever before for ARIN to spend time on netblock sanitization. I do think that ARIN should inform the new netblock owner if it was previously owned or not. But if ARIN tried to start cleaning up a netblock before releasing it, there would be no end to it. How could they check against the probably hundreds of thousands private blocklist? They could implement a process by which they announce to a mailing list of DNSBL providers that a given assignment has been returned to the RIR and that it should be cleansed from all DNSBLs. You mean like this? http://lists.arin.net/pipermail/arin-issued/2009-September/000270.html -M
Re: Repeated Blacklisting / IP reputation
On Sep 11, 2009, at 6:52 PM, Martin Hannigan wrote: I honestly don't think that it's up to them to create a set-aside either, hence my comment about behind the scenes activities. I appreciate you detailing that, but I honestly don't think it matters since as you mentioned you get accused of this all of the time. I would expect that ICANN would not only follow the rules, but safeguard them as well. The RIR CEO's told the IANA to use their best judgement in making the /8 assignments. This is exactly what happens with each assignment today in any case, and would have been the same result without that feedback to IANA, i.e., what would normally have been a behind the scenes implementation issue has now been publicly detailed, and I, for one, thank the IANA for their clear and timely communications on this matter. Numbering policy usually goes to the members of each of the RIR communities, just as the IANA to RIR policy did. The algorithm itself is great. The set-aside is the problem. This is not formation of global Internet numbering policy, it's implementation of the existing policy regarding IANA to RIR /8 block assignments. Regardless, the global nature of the Internet means that we'll all deal with connectivity issues with these blocks once they're allocated. Any and all efforts that the networking community can take now to get these blocks cleaned up now would be most helpful. /John John Curran President and CEO ARIN
Re: Repeated Blacklisting / IP reputation
On Wed, Sep 9, 2009 at 11:30 PM, Leo Vegoda leo.veg...@icann.org wrote: On Sep 9, 2009, at 7:18 PM, Alex Lanstein wrote: Along the same lines, I noticed that the worst Actor in recent memory (McColo - AS26780) stopped paying their bills to ARIN and their addresses have been returned to the pool. It's my opinion that a very select number of CIDR blocks (another example being the ones belonging to Cernel/InternetPath/Atrivo/etc, if it were ever fully extinguished) are, and forever will be, completely toxic and unusable to any legitimate enterprise. Arguments could be made that industry blacklists can and should be more flexible, but from the considerably more innocuous case in this thread, that is apparently not the modus operandi Putting these addresses back into use does not mean that they have to be allocated to networks where they'll number mail servers. ARIN staff is doubtless aware of the history of these blocks and will presumably do their best to allocate them to networks that aren't intended to host mail servers. to quote bmanning.. they may even be put into service on a network that is not 'the internet'. Though I think Alex's idea isn't without merit, perhaps as a stage between 'de-allocate from non-payer' and 'allocate to new payer'. (perhaps only for blocks meeting some set of criteria, yet to be determined/discussed) -Chris
Re: Repeated Blacklisting / IP reputation
On Wed, Sep 9, 2009 at 11:48 PM, Mark Andrews ma...@isc.org wrote: skip a note about isc having quite a few legacy blocks Note we all could start using IPv6 and avoid this problem altogether. There is nothing stopping us using IPv6 especially for MTA's. that'd solve the spam problem... for a while at least. (no ipv6 traffic == no spam) -Chris (yes, I'm yanking mark's chain some)
Re: Repeated Blacklisting / IP reputation
Joe == Joe Greco jgr...@ns.sol.net writes: Joe Show me ONE major MTA which allows you to configure an expiration Joe for an ACL entry. Any MTA which supports using an sql db as its backend. Postfix is a fine example. You just define the table and the query to either have an until column, or have a column with the timestamp of when the entry was added and have the query ignore rows which are older than some given time. And with postfix, using its sql proxy capability, using a sql backend is fully performant. -JimC -- James Cloos cl...@jhcloos.com OpenPGP: 1024D/ED7DAEA6
Re: Repeated Blacklisting / IP reputation
Joe == Joe Greco jgr...@ns.sol.net writes: Joe Show me ONE major MTA which allows you to configure an expiration Joe for an ACL entry. Any MTA which supports using an sql db as its backend. Postfix is a fine example. You just define the table and the query to either have an until column, or have a column with the timestamp of when the entry was added and have the query ignore rows which are older than some given time. And with postfix, using its sql proxy capability, using a sql backend is fully performant. So, you agree, MTA's do not implement this functionality. It's obviously possible to make it happen through shell scripting, database tricks, etc., but the point was that if this was commonly desired, then MTA's would be supporting it directly. It isn't commonly desired, most people just block forever. It never ceases to amaze me how technical people so often easily miss the point. :-) ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
Joe == Joe Greco jgr...@ns.sol.net writes: Joe So, you agree, MTA's do not implement this functionality. It's Joe obviously possible to make it happen through shell scripting, Joe database tricks, No, I do not agree. The sql backend is part of the MTA; features added by offering a sql backend for tables of this sort (I'd use a cidr access restriction in postfix) are still features of the MTA. And actually using the power of sql when using sql is not a trick; rather it is the /point/. IOW, the MTA is the sum of its parts; when using sql lookups the db is part of the MTA. -JimC -- James Cloos cl...@jhcloos.com OpenPGP: 1024D/ED7DAEA6
Re: Repeated Blacklisting / IP reputation
Joe == Joe Greco jgr...@ns.sol.net writes:
Joe So, you agree, MTA's do not implement this functionality. It's
Joe obviously possible to make it happen through shell scripting,
Joe database tricks,
No, I do not agree.
The sql backend is part of the MTA; features added by offering a sql
backend for tables of this sort (I'd use a cidr access restriction
in postfix) are still features of the MTA.
And actually using the power of sql when using sql is not a trick;
rather it is the /point/.
IOW, the MTA is the sum of its parts; when using sql lookups the db
is part of the MTA.
By that argument, anything else that you install that augments the
functionality of your MTA in some manner is part of your MTA. Since
DSPAM hooks into Postfix, clearly Postfix offers Bayesian filtering,
and since ClamAV hooks in, clearly Postfix offers spam filtering, and
since you can use LogReport to manage its logs, clearly Postfix offers
reporting via an HTTP interface, and since I find it convenient to have
a shell on a mail server, when I install tcsh or zsh, that's also an
offering by Postfix.
No.
You show me a line in Postfix's ACL code that reads to the effect of
if (expiryfield time(NULL)) {
accept_message;
}
and then that's PART of the MTA. Otherwise, it's an add-on of some sort.
Given that the point I was making was about capabilities *included* in
the MTA, and given that I *said* you could add on such functions, it's
kind of silly to try to confuse the issue in this manner.
In other words, if it doesn't compile out of the box with it, that's what
I was talking about, and that's the point. No add-ons, no enhancements.
We already know that something can be *added* to help the MTA implement
such a feature; that's obvious to everyone. However, it isn't commonly
done, and dlr posted stats indicating that a significant percentage of
spam-spewing IP addresses would continue to do so for *years*. As a
result, mail admins typically throw IP's in ACL's for something that
approaches *forever*.
The point was that MTA's don't support anything else by default, that
such a feature isn't in demand, and that the spam database analysis
supports this as a not entirely unreasonable state of affairs.
Further, since it is relatively unlikely, statistically speaking, that
any particular IP address
I'm not interested in playing semantic games about what constitutes
an MTA. I *am* interested in the general problem of outdated rules
of any sort that block access to reallocated IP space; this is a real
operational problem, both to recipients of such space, and to sites who
have blocked such space.
My tentative conclusion is that there is no realistic solution to the
overall problem. Even within a single autonomous system, there usually
isn't a comprehensive single unified method for denying access to
services; you might have separate lists for IP in general (bogons),
access to mail systems (DNSBL's and local rules derived from bad
experiences), rules for access to various devices and services, rules
added to block syn floods from/to, etc., etc., etc. And all of the
systems to implement these rules are more or less disjoint.
The concept of virgin IPv4 space is going to be a memory soon.
... JG
--
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
RE: Repeated Blacklisting / IP reputation
With scarcity of IPv4 addresses, organizations are more desperate than ever to receive an allocation. If anything, there's more of a disincentive than ever before for ARIN to spend time on netblock sanitization. I do think that ARIN should inform the new netblock owner if it was previously owned or not. But if ARIN tried to start cleaning up a netblock before releasing it, there would be no end to it. How could they check against the probably hundreds of thousands private blocklist? Frank -Original Message- From: JC Dill [mailto:jcdill.li...@gmail.com] Sent: Wednesday, September 09, 2009 5:40 PM To: NANOG list Subject: Re: Repeated Blacklisting / IP reputation snip They can (and IMHO should) determine the state it is in before they reallocate it. What happens next is obviously unpredictable but in reality an IP that isn't being blocked today and isn't being used (by anyone) is highly unlikely to be widely blocked between today and the day ARIN releases it for allocation to a new entity. They can hold IPs that are not suitable for re-allocation, or at least make the status of the IPs known to the new entity before asking the entity to take on the IP block, and perhaps offering a fee discount for tainted addresses. (Some users may not care if the IPs are tainted, if, for instance they plan to use the IPs for a DUL pool. I have a friend who gets $5 off his cell phone bill because he has a phone number that starts with 666 - a number that many people prefer to avoid but which works fine for his purposes and he's quite happy to get the discount. :-) snip ARIN shouldn't allocate previously allocated IPs until they know the IPs are not widely blocked. Or to *at the very least* ARIN should disclose what they know about the IP space before they make it someone else's problem, and give the requesting entity an option to request a new/clean/unused/unblocked IP block instead. snip jc
Re: Repeated Blacklisting / IP reputation
Frank Bulk wrote: With scarcity of IPv4 addresses, organizations are more desperate than ever to receive an allocation. Factual evidence that pi allocation is in fact hard to obtain would be required to support that statement. The fact of the matter is if you have a legitimate application congruent with current policy you'll get your addresses just like you would last year. Now if your business is contingent on the availability of pi addressing resources obviously you have a fiduciary responsibility to address that problem in short order. If anything, there's more of a disincentive than ever before for ARIN to spend time on netblock sanitization. This whole thread seems to be about shifting (I.E. by externalizing) the costs of remediation. presumably the entities responsible for the poor reputation aren't likely to pay... So heck, why not ARIN? perhaps because it's absurd on the face of it? how much do my fees go up in order to indemnify ARIN against the cost of a possible future cleanup? how many more staff do they need? Do I have to buy prefix reputation insurance as contingent requirement for a new direct assignment? I do think that ARIN should inform the new netblock owner if it was previously owned or not. We've got high quality data extending back through a least 1997 on what prefixes have been advertised in the DFZ, and of course from the ip reputation standpoint it doesn't so much matter if something was assigned, but rather whether it was ever used. one assumes moreover that beyond a certain point in the not too distant future it all will have been previously assigned (owned is the wrong word). But if ARIN tried to start cleaning up a netblock before releasing it, there would be no end to it. How could they check against the probably hundreds of thousands private blocklist? Note that they can't insure routability either, though as a community we've gotten used to testing for stale bogon filters. Frank -Original Message- From: JC Dill [mailto:jcdill.li...@gmail.com] Sent: Wednesday, September 09, 2009 5:40 PM To: NANOG list Subject: Re: Repeated Blacklisting / IP reputation snip They can (and IMHO should) determine the state it is in before they reallocate it. What happens next is obviously unpredictable but in reality an IP that isn't being blocked today and isn't being used (by anyone) is highly unlikely to be widely blocked between today and the day ARIN releases it for allocation to a new entity. They can hold IPs that are not suitable for re-allocation, or at least make the status of the IPs known to the new entity before asking the entity to take on the IP block, and perhaps offering a fee discount for tainted addresses. (Some users may not care if the IPs are tainted, if, for instance they plan to use the IPs for a DUL pool. I have a friend who gets $5 off his cell phone bill because he has a phone number that starts with 666 - a number that many people prefer to avoid but which works fine for his purposes and he's quite happy to get the discount. :-) snip ARIN shouldn't allocate previously allocated IPs until they know the IPs are not widely blocked. Or to *at the very least* ARIN should disclose what they know about the IP space before they make it someone else's problem, and give the requesting entity an option to request a new/clean/unused/unblocked IP block instead. snip jc
RE: Repeated Blacklisting / IP reputation
and then that's PART of the MTA. Otherwise, it's an add-on of some sort. Given that the point I was making was about capabilities *included* in the MTA, and given that I *said* you could add on such functions, it's kind of silly to try to confuse the issue in this manner. CommuniGate Pro supports time limited blacklisting, at least for Ips it blacklists itself based on protocol violations c.
Re: Repeated Blacklisting / IP reputation
Peter Beckman wrote: On Thu, 10 Sep 2009, Mark Andrews wrote: What a load of rubbish. How is ARIN or any RIR/LIR supposed to know the intent of use? Why don't we just blacklist everything and only whitelist those we know are good? Because the cost of determining who is good and who is not has a great cost. If you buy an IP block, regardless of your intent, that IP block should not have the ill-will of the previous owner passed on with it. You don't buy ip blocks or at least not from ARIN. Among other things that ARIN does not guarantee is routability. If the previous owner sucked, the new owner should have the chance to use that IP block without restriction until they prove that they suck, at which point it will be blocked again. That system seems to work well enough: blacklist blocks when they start do be evil, according to your own (you being the neteng in charge) definition of evil. ARIN needs to be impartial. If they are going to sell the block, they should do their best to make a coordinated effort to make sure the block is as unencumbered as possible. I get that there is a sense that ARIN needs to do more due dilligence to determine if the receiving party is worthy of that block, but I'm not aware of the process, and from the grumblings it doesn't seem like fun. Note we all could start using IPv6 and avoid this problem altogether. Because as we know IPv6 space is inexhaustable. Just like IPv4 was when it began its life. ;-) That won't avoid the problem, it will simply put the problem off until it rears its head again. I'm sure that IPv6 space will be more easily gotten until problems arise, and in a few years (maybe decades, we can put this problem on our children's shoulders), we'll be back where we are now -- getting recycled IP space that is blocked or encumbered due to bad previous owners. Beckman --- Peter Beckman Internet Guy beck...@angryox.com http://www.angryox.com/ ---
Re: Repeated Blacklisting / IP reputation
Benjamin Billon wrote: Why don't we just blacklist everything and only whitelist those we know are good? snip Note we all could start using IPv6 and avoid this problem altogether. snip Yeah. When ISP will start receiving SMTP traffic in IPv6, they could start to accept whitelisted senders only. I've been reciveving smtp traffic including spam on ipv6 since 2001. IPv6 emails == clean Utopian thought?
Re: Repeated Blacklisting / IP reputation
Marty, On Sep 10, 2009, at 2:45 PM, Martin Hannigan wrote: Not sure when ICANN got into the business of economic bailouts, ?? The blog posting implies it: AfriNIC and LACNIC have fewest IPv4 /8s and service the regions with the most developing economies. We decided that those RIRs should have four of the easiest to use /8s reserved for them. The economies term used here is essentially synonymous with countries. The decision IANA made (which is, of course, always reversible until the last /8s are allocated) is in keeping with RIR practices regarding treatment of LACNIC and AfriNIC in global allocation issues. There is also a possible unintended consequence. If v4 address space markets do end up being legitimized (I do believe that they will FWIW) ICANN is in effect declaring one class of space more valuable than another an arbitrarily assigning that value. ICANN is not declaring value of anything. All we are doing is trying to distribute the remaining /8s in a way that can be publicly verified that we have no bias in how /8s are allocated at the same time as trying to minimize the pain experienced by the recipients the /8s. Or are you unhappy that LACNIC and AfriNIC have 2 /8s from the least tainted pools? There is currently a global policy that the RIR's and ICANN agreed to that defines the allocation of /8's from IANA to RIR's. That policy doesnt include a set-aside and I think that arbitrarily adding one is not in the spirit of cooperation. The global policy for IPv4 address allocation does not specify how IANA selects the addresses it assigns to the RIRs. IANA has used different algorithms in the past. What IANA is doing now is described in the blog posting I referenced. It's possible that not everything is above the table as well. Actually, no. The whole point in publishing the algorithm IANA is using in allocating /8s is to allow anyone to verify for themselves we are following that algorithm. I think that the perception is reality here though. ICANN has arbitrarily created process that impacts RIR's unequally. To me, that's unfair. As stated, we followed existing RIR practices regarding treatment of LACNIC and AfriNIC. Oddly, the RIR CEOs were happy with the algorithm when we asked them about it. Question is -- do a few /8's really matter? Sure. An they'll matter more as the IPv4 pool approaches exhaustion. That's why IANA has published the algorithm by which allocations are made. The goal is to forestall (or at least help defend from) the inevitable accusations of evil doing folks accuse ICANN of all the time (e.g., your message). Regards, -drc
Re: Repeated Blacklisting / IP reputation
On Fri, Sep 11, 2009 at 4:23 PM, David Conrad d...@virtualized.org wrote: Marty, It's possible that not everything is above the table as well. Actually, no. The whole point in publishing the algorithm IANA is using in allocating /8s is to allow anyone to verify for themselves we are following that algorithm. Sorry, poor wording on my part. See below. I think that the perception is reality here though. ICANN has arbitrarily created process that impacts RIR's unequally. To me, that's unfair. As stated, we followed existing RIR practices regarding treatment of LACNIC and AfriNIC. Oddly, the RIR CEOs were happy with the algorithm when we asked them about it. I honestly don't think that it's up to them to create a set-aside either, hence my comment about behind the scenes activities. I appreciate you detailing that, but I honestly don't think it matters since as you mentioned you get accused of this all of the time. I would expect that ICANN would not only follow the rules, but safeguard them as well. Numbering policy usually goes to the members of each of the RIR communities, just as the IANA to RIR policy did. The algorithm itself is great. The set-aside is the problem. I'd be happy with the algorithm and all of the space. It would be more fair to us all and not appear as a cost shifting or potential windfall. Best, -M -- Martin Hannigan mar...@theicelandguy.com p: +16178216079 Power, Network, and Costs Consulting for Iceland Datacenters and Occupants
Re: Repeated Blacklisting / IP reputation
On Wed, Sep 09, 2009 at 04:13:18PM -0700, Jay Hennigan wrote: JC Dill wrote: As for a role account, there is postmaster. I would think that the best hope in the real world, rather than an autoresponder would be an RFC that clearly defines text accompanying an SMTP rejection notice triggered by a blocklist, detailing the blocklist and contact for removal. Perhaps encouraging those who code MTAs and DNSBL hooks into them to include such in the configuration files would be a good start. That would be very useful. Many of those small lists return 'Unknown user' rather than an actual blacklist message. A url where one could get reason (meaning headers) for the block would be even better. If they don't admit that it's a block, it's hard to do much more than tell the user to contact the recipient via some other channel and have *them* contact their support system. -- Dave - Nobody believed that I could build a space station here. So I built it anyway. It sank into the vortex. So I built another one. It sank into the vortex. The third station burned down, fell over then sank into the vortex. The fourth station just vanished. And the fifth station, THAT stayed!
Re: Repeated Blacklisting / IP reputation
Hi Tom (and NANOG), You may be interested in an alternative approach, motivated by the very problem you are facing (see below). Our system, SNARE, develops IP reputation automatically based on a combination of network features. We'll discuss the pros and cons of this approach at MAAWG. The additional information that SNARE provides might be helpful. -Nick Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang Hao, Nadeem Ahmed Syed, Nick Feamster, Alexander Gray, Sven Krasser Usenix Security '09, Montreal, Canada, August 2009 Users and network administrators need ways to filter email messages based primarily on the reputation of the sender. Unfortunately, conventional mechanisms for sender reputation -- notably, IP blacklists -- are cumbersome to maintain and evadable. This paper investigates ways to infer the reputation of an email sender based solely on network-level features, without looking at the contents of a message. First, we study first-order properties of network-level features that may help distinguish spammers from legitimate senders. We examine features that can be ascertained without ever looking at a packet's contents, such as the distance in IP space to other email senders or the geographic distance between sender and receiver. We derive features that are lightweight, since they do not require seeing a large amount of email from a single IP address and can be gleaned without looking at an email's contents -- many such features are apparent from even a single packet. Second, we incorporate these features into a classification algorithm and evaluate the classifier's ability to automatically classify email senders as spammers or legitimate senders. We build an automated reputation engine, SNARE, based on these features using labeled data from a deployed commercial spam-filtering system. We demonstrate that SNARE can achieve comparable accuracy to existing static IP blacklists: about a 70% detection rate for less than a 0.3% false positive rate. Third, we show how SNARE can be integrated into existing blacklists, essentially as a first-pass filter. http://gtnoise.net/pub/index.php?detail=14 On Tue, Sep 8, 2009 at 4:58 PM, Tom Pipes tom.pi...@t6mail.com wrote: I am amazed with the amount of thoughtful comments I have seen, both on and off list. It really illustrates that people are willing to try to help out, but there is an overall lack of clear direction on how to improve things. Most of us seem to adopt that which has always just worked for us. Don't get me wrong, I'm sure there are a lot of improvements/mods going on with RBL operators in terms of the technology and how they choose who to block. I'm also certain that most of the carriers are doing their best to follow RFCs, use e-mail filtering, and perform deep packet inspection to keep themselves off of the lists. AND there seems to be some technologies that were meant to work, and cause their own sets of problems (example: allowing the end user to choose what is considered spam and blacklisting based on that). As was said before, it's not the WHY but rather how can we fix it if it's broke. The large debate seems to revolve around responsibility, or lack thereof. In our case, we are the small operator who sits in the sidelines hoping that someone larger than us, or more influential has an opinion. We participate in lists, hoping to make a difference and contribute, knowing that in a lot of cases, our opinion is just that: an opinion. I suppose that could spark a debate about joining organizations (who shall go nameless here), power to the people, etc. It seems as though a potential solution *may* revolve around ARIN/IANA having the ability to communicate an authoritative list of reassigned IP blocks back to the carriers. This could serve as a signal to remove a block from the RBL, but I'm sure there will be downfalls with doing this as well. In my specific case, I am left with a legacy block that I have to accept is going to be problematic. Simply contacting RBL operators is just not doing the trick. Most of the e-mails include links or at least an error code, but some carriers just seem to be blocking without an error, or even worse, an ACL... We will continue to remove these blocks as necessary, reassign IPs from other blocks where absolutely necessary, and ultimately hope the problem resolves itself over time. Thanks again for the very thoughtful and insightful comments, they are greatly appreciated. Regards, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com - Original Message - From: Tom Pipes tom.pi...@t6mail.com To: nanog@nanog.org Sent: Tuesday, September 8, 2009 9:57:58 AM GMT -06:00 US/Canada Central Subject: Repeated Blacklisting / IP reputation Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for
Re: Repeated Blacklisting / IP reputation
On Thu, 10 Sep 2009, Mark Andrews wrote: What a load of rubbish. How is ARIN or any RIR/LIR supposed to know the intent of use? Why don't we just blacklist everything and only whitelist those we know are good? Because the cost of determining who is good and who is not has a great cost. If you buy an IP block, regardless of your intent, that IP block should not have the ill-will of the previous owner passed on with it. If the previous owner sucked, the new owner should have the chance to use that IP block without restriction until they prove that they suck, at which point it will be blocked again. That system seems to work well enough: blacklist blocks when they start do be evil, according to your own (you being the neteng in charge) definition of evil. ARIN needs to be impartial. If they are going to sell the block, they should do their best to make a coordinated effort to make sure the block is as unencumbered as possible. I get that there is a sense that ARIN needs to do more due dilligence to determine if the receiving party is worthy of that block, but I'm not aware of the process, and from the grumblings it doesn't seem like fun. Note we all could start using IPv6 and avoid this problem altogether. Because as we know IPv6 space is inexhaustable. Just like IPv4 was when it began its life. ;-) That won't avoid the problem, it will simply put the problem off until it rears its head again. I'm sure that IPv6 space will be more easily gotten until problems arise, and in a few years (maybe decades, we can put this problem on our children's shoulders), we'll be back where we are now -- getting recycled IP space that is blocked or encumbered due to bad previous owners. Beckman --- Peter Beckman Internet Guy beck...@angryox.com http://www.angryox.com/ ---
Re: Repeated Blacklisting / IP reputation
Why don't we just blacklist everything and only whitelist those we know are good? snip Note we all could start using IPv6 and avoid this problem altogether. snip Yeah. When ISP will start receiving SMTP traffic in IPv6, they could start to accept whitelisted senders only. IPv6 emails == clean Utopian thought?
Re: Repeated Blacklisting / IP reputation
On Thu, Sep 10, 2009 at 04:42:13PM +0200, Benjamin Billon wrote: Why don't we just blacklist everything and only whitelist those we know are good? snip Note we all could start using IPv6 and avoid this problem altogether. snip Yeah. When ISP will start receiving SMTP traffic in IPv6, they could start to accept whitelisted senders only. IPv6 emails == clean Utopian thought? abt 8 years too late... --bill
Re: Repeated Blacklisting / IP reputation
Benjamin Billon wrote: Why don't we just blacklist everything and only whitelist those we know are good? snip Note we all could start using IPv6 and avoid this problem altogether. snip Yeah. When ISP will start receiving SMTP traffic in IPv6, they could start to accept whitelisted senders only. IPv6 emails == clean Utopian thought? Are you not receiving SMTP traffic via IPv6 yet? Received: from s0.nanog.org ([IPv6:2001:48a8:6880:95::20]) - Kevin
Re: Repeated Blacklisting / IP reputation
On Thu, 10 Sep 2009, Benjamin Billon wrote: Why don't we just blacklist everything and only whitelist those we know are good? snip Note we all could start using IPv6 and avoid this problem altogether. snip Yeah. When ISP will start receiving SMTP traffic in IPv6, they could start to accept whitelisted senders only. IPv6 emails == clean Utopian thought? My statement about blacklisting everything was sarcastic. Clearly blacklisting everything and whitelisting individual blocks is not a viable, reasonable nor cost-effective option. Clearly I also suck at conveying sarcasm via email. :-) Beckman --- Peter Beckman Internet Guy beck...@angryox.com http://www.angryox.com/ ---
Re: Repeated Blacklisting / IP reputation
You're not Hotmail =)
Re: Repeated Blacklisting / IP reputation
On Wed, 09 Sep 2009 20:30:02 PDT, Leo Vegoda said: Putting these addresses back into use does not mean that they have to be allocated to networks where they'll number mail servers. ARIN staff is doubtless aware of the history of these blocks and will presumably do their best to allocate them to networks that aren't intended to host mail servers. Those streaming video servers in that returned /24 are going to work *real* well talking to a network that implemented the block as a null route rather than a port-25 block. pgpTDcdvozLS7.pgp Description: PGP signature
Re: Repeated Blacklisting / IP reputation
Because the cost of determining who is good and who is not has a great cost. If you buy an IP block, regardless of your intent, that IP block should not have the ill-will of the previous owner passed on with it. Might as well be the end of discussion, right there, then, because what you're suggesting suggests no grasp of the real world. If the previous owner sucked, the new owner should have the chance to use that IP block without restriction until they prove that they suck, at which point it will be blocked again. That system seems to work well enough: blacklist blocks when they start do be evil, according to your own (you being the neteng in charge) definition of evil. What you just described doesn't implement what you claim, at all. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
On Sep 9, 2009, at 8:41 PM, Martin Hannigan wrote: Not sure when ICANN got into the business of economic bailouts, ?? but the mechanism that ICANN has defined seems patently unfair. RFC 2777 is unfair? Or are you unhappy that LACNIC and AfriNIC have 2 /8s from the least tainted pools? Regards, -drc
Re: Repeated Blacklisting / IP reputation
On Thu, Sep 10, 2009 at 4:21 PM, David Conrad d...@virtualized.org wrote: On Sep 9, 2009, at 8:41 PM, Martin Hannigan wrote: Not sure when ICANN got into the business of economic bailouts, ?? The blog posting implies it: AfriNIC and LACNIC have fewest IPv4 /8s and service the regions with the most developing economies. We decided that those RIRs should have four of the easiest to use /8s reserved for them. There is also a possible unintended consequence. If v4 address space markets do end up being legitimized (I do believe that they will FWIW) ICANN is in effect declaring one class of space more valuable than another an arbitrarily assigning that value. but the mechanism that ICANN has defined seems patently unfair. RFC 2777 is unfair? Or are you unhappy that LACNIC and AfriNIC have 2 /8s from the least tainted pools? I don't have a comment on the RFC. There is currently a global policy that the RIR's and ICANN agreed to that defines the allocation of /8's from IANA to RIR's. That policy doesnt include a set-aside and I think that arbitrarily adding one is not in the spirit of cooperation. I think that it's good that ICANN is being proactive, but I also think that it's bad that they chose this to be proactive about. It's possible that not everything is above the table as well. I think that the perception is reality here though. ICANN has arbitrarily created process that impacts RIR's unequally. To me, that's unfair. Question is -- do a few /8's really matter? In the end game, I think that they do all considered. Best, Marty -- Martin Hannigan mar...@theicelandguy.com p: +16178216079 Power, Network, and Costs Consulting for Iceland Datacenters and Occupants
Re: Repeated Blacklisting / IP reputation
On 09/09/2009 8:48, Mark Andrews ma...@isc.org wrote: [...] What a load of rubbish. How is ARIN or any RIR/LIR supposed to know the intent of use? In my limited experience, requesting address space from ARIN involved describing what I would be doing with it. YMMV. Leo
Re: Repeated Blacklisting / IP reputation
--- leo.veg...@icann.org wrote: In my limited experience, requesting address space from ARIN involved describing what I would be doing with it. YMMV. - That's the easy part of the process. Proof of what you did with what you already have assigned to you is the hard part. scott
Re: Repeated Blacklisting / IP reputation
bmann...@vacation.karoshi.com wrote: sounds like domain tasting to me. Oops! Oh yeah. Spammer gets an allocation... Well, if that netblock was clean before, it sure isn't now! May I please have another? Lather, rinse, repeat. -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: Repeated Blacklisting / IP reputation
On 08/09/09 21:34, Joe Greco wrote: Show me ONE major MTA which allows you to configure an expiration for an ACL entry. This is fairly trivial to do with Exim by storing your acl entries in a database or directory with a field/attribute for expiry, and an appropriate router configuration. No doubt you could implement this using a small script for any MTA. The upside of using a db/ldap backend is that it makes it easy to inter-operate with other things like your nms.
Re: Repeated Blacklisting / IP reputation
Show me ONE major MTA which allows you to configure an expiration for an ACL entry. The problem with your opinion, and it's a fine opinion, and it's even a good opinion, is that it has very little relationship to the tools which are given to people in order to accomplish blocking. Kind of the question I was contemplating in my other message of minutes ago. If people were given an option to block this IP for 30 minutes, 24 hours, 30 days, 12 months, 5 years, or forever - I wonder how many people would just shrug and click forever. This may lead to the discovery of another fundamental disconnect - or two. Sigh. ... JG A cron job/schedule task with a script that removes said line would most likely do wonderous things for you. I could see a comment before each listing with a time/date that you use some regex fu on to figure out how long it was there and how long it should be there for. Simple! You could also automate it with a web frontend for noobs so they don't have to manually edit configuration files. You /COMPLETELY/ missed the point. If this was something that people felt was truly useful, then there would be support for something like this. I mean, we've only had about 15 years of spam-as-a-real-problem on the Internet. The perception by most admins is that when you block someone, you want to block them for a Really Long Time. If this wasn't true, then there would likely be an automatic feature built in to MTA ACL entries to expire. I didn't say you /couldn't/ do it. The problem is that the average spam spewer is a long-term thing, so when you ACL off a host, you've probably deemed the sender to be of no significant value to you, and you're not expecting that they're suddenly going to become whitehat in two weeks, or even six months. Therefore, there's no default support built into MTA's for this, because it /doesn't/ do anything wonderous for you. I would agree that in the best case, we would want a default behaviour of ACL removal when an IP block is reallocated by the RIR, but I don't see an easy way to get there as a default behaviour of an MTA. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
bmann...@vacation.karoshi.com wrote: sounds like domain tasting to me. Oops! Oh yeah. Spammer gets an allocation... Well, if that netblock was clean before, it sure isn't now! May I please have another? Lather, rinse, repeat. THAT would probably be easy enough to detect; RIR simply checks to see if new DNSBL entries had appeared, and refuses to trade in the block if any do. You may need a few more refinements too. I don't think it's technically unworkable, if tackled correctly. But it also leaves some questions, such as what ARIN is expected to do with the toxic wastelands left behind by spammers. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
[In the message entitled Re: Repeated Blacklisting / IP reputation on Sep 8, 14:34, Joe Greco writes:] there is a fundamental disconnect here. the IP space is neutral. it has no bias toward or against social behaviours. its a tool. the actual/real target here are the people who are using these tools to be antisocial. blacklisting IP space is always reactive and should only beused in emergency and as a -TEMPORARY- expedient. IMHO of course., YMMV. If people were given an option to block this IP for 30 minutes, 24 hours, 30 days, 12 months, 5 years, or forever - I wonder how many people would just shrug and click forever. This may lead to the discovery of another fundamental disconnect - or two. IP address space is neutral, but the operators of the space either permit, or deny, the social behaviour which comes from these spaces. For what it's worth, I just completed a study of about 5 years of data on spam. I looked at 100,000,000 IP addresses which had sent me spam. The median duration of sending was 300 days. There was a pronounced peak at 2-3 years of about 30%. The vast majority was more than 30 days. forever is pretty close to right, based on current behaviour. --
Re: Repeated Blacklisting / IP reputation
John, ARIN's role as the entity engaged in legal contractual relationship with the previous owners of the space puts it in the position to insert enforceable contract clauses to deter and/or mitigate graffiti in allocations. Policy proposals probably are not required for this. Space originally from outside ARIN, thats another kettle of fish. ARIN is also in the position to refuse allocations for entities who dont clean up after themselves. Policy likely required. And finally, if this problem continues to worsen (as it likely will when greenfield becomes scarce), a viable business opportunity should emerge for reputable organizations to do cleanup on behalf of the new owners, for a reasonable fee/retainer and after suitable financial/contractual guarantees. Cost of business, efficiency of scale and all that. Perhaps the bill could even be sent to the previous owners. Operationally, I dont see how the problem can be mitigated solely by those who are already informed. Joe John Curran wrote: Folks - It appears that we have a real operational problem, in that ARIN does indeed reissue space that has been reclaimed/returned after a hold-down period, and but it appears that even once they are removed from the actual source RBL's, there are still ISP's who are manually updating these and hence block traffic much longer than necessary. I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? Thanks! /John John Curran President and CEO ARIN On Sep 8, 2009, at 11:16 AM, Ronald Cotoni wrote: Tom Pipes wrote: Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for lack of a better word) since we obtained it. It seems like every customer we have added has had repeated issues with being blacklisted by DUL and the cable carriers.
Re: Repeated Blacklisting / IP reputation
John, ARIN's role as the entity engaged in legal contractual relationship with the previous owners of the space puts it in the position to insert enforceable contract clauses to deter and/or mitigate graffiti in allocations. That's complicated. How do you define graffiti? Just for starters. Given that even a whitehat network can generate occasional complaints, and most commercial networks generate various levels of cruft, would you consider it graffiti if a block of IP space assigned to a hotel wifi network in Seattle got itself permanently ACL'ed by a college in Miami, when someone inadvertently omitted the port 25 filter, and as a result, the mail admins in Miami judged that the likelihood of ever receiving legitimate mail from there was about 0.0001%? How would you even know? Policy proposals probably are not required for this. Space originally from outside ARIN, thats another kettle of fish. ARIN is also in the position to refuse allocations for entities who dont clean up after themselves. Policy likely required. How exactly do you do that? Spammers don't mind submitting fraudulent applications. How does ARIN tell that SpamNetA is actually the same operation as FooIspB, even though they might be legally registered as different companies? And finally, if this problem continues to worsen (as it likely will when greenfield becomes scarce), a viable business opportunity should emerge for reputable organizations to do cleanup on behalf of the new owners, for a reasonable fee/retainer and after suitable financial/contractual guarantees. Cost of business, efficiency of scale and all that. Perhaps the bill could even be sent to the previous owners. That's likely to stand up in court. Not. Operationally, I dont see how the problem can be mitigated solely by those who are already informed. I agree that it's problematic. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
RE: Repeated Blacklisting / IP reputation
Right on point -- we have a long list of manually entered netblocks in our spam appliance's blacklist that we've accumulated over time. Besides the mistakes we've made, we've had to delist perhaps 5 over the last 2 years, none due to ARIN reallocations. Most times it's our customer calling our helpdesk and saying I can't get an e-mail from so-and-so. There's a strong (time resource) disincentive for us to review netblocks and then delist them. Ideally our spam appliance vendor would show us a top ten of non-hit netblocks and we would remove them then (i.e. if no one has hit an IP in that range for a month, the spammer has probably moved on), or as another person suggested, just have the spam appliance age them out (change the action applied from blocked to do nothing. One of the potential community-based approaches would be to have a hosted RBL, with a 'view' for each SP or enterprise. That is, each RBL would be unique, but if I trusted organization B, I could request to use their RBL entries, too. Rather than managing a manual list, it would be managed on the web with more management tools: - search by date added, size of netblock, hits, etc. - auto expiration/aging - notification if netblock assigned to a new owner - comparison against other RBLs (no use having it on my company's RBL is Spamhaus has added it) than an admin of a small operation would likely have. Contact info could be made available, mechanism to request delisting, etc. Frank -Original Message- From: Jay Hennigan [mailto:j...@west.net] Sent: Tuesday, September 08, 2009 1:14 PM To: John Curran Cc: nanog@nanog.org Subject: Re: Repeated Blacklisting / IP reputation John Curran wrote: Folks - It appears that we have a real operational problem, in that ARIN does indeed reissue space that has been reclaimed/returned after a hold-down period, and but it appears that even once they are removed from the actual source RBL's, there are still ISP's who are manually updating these and hence block traffic much longer than necessary. I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? I don't think there is an excellent reason, more likely inertia and no real incentive to put forth the effort to proactively remove addresses. Many ISPs and organizations have their own private blocklists not associated with the widely known DNSBLs. Typically during or immediately after a spam run the mail administrator will manually add offending addresses or netblocks. Spamtrap hits may do this automatically. There isn't any real incentive for people to go back and remove addresses unless they're notified by their own customers that legitimate mail coming from those addresses is being blocked. Because these blocklists are individually maintained, there is no central registry or means to clean them up when an IP assignment changes. To make matters worse, some organizations may simply ACL the IP space so that the TCP connection is never made in the first place (bad, looks like a network problem rather than deliberate filtering), some may drop it during SMTP with no clear indication as to the reason (less bad, as there is at least a hint that it could be filtering), and some may actually accept the mail and then silently discard it (worst). In addition there are several DNSBLs with different policies regarding delisting. Some just time out after a period of time since abuse was detected. Some require action in the form of a delisting request. Some require a delisting request and a time period with no abuse. Some (the old SPEWS list) may not be easily reached or have well defined policies. In meatspace, once a neighborhood winds up with a reputation of being rife with drive-by shootings, gang activity and drug dealing it may take a long time after the last of the graffiti is gone before some cab drivers will go there. -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
RE: Repeated Blacklisting / IP reputation
What's to stop spammers from doing this to cycle through blocks in rapid-fashion? This proposal seems easily abusable to me. - S From: Peter Beckman [beck...@angryox.com] Sent: Tuesday, September 08, 2009 10:04 PM To: Tom Pipes Cc: nanog@nanog.org Subject: Re: Repeated Blacklisting / IP reputation How about a trial period from ARIN? You get your IP block, and you get 30 days to determine if it is clean or not. Do some testing, check the blacklists, do some magic to see if there are network-specific blacklists that might prevent your customers from sending or receiving email/web/other connections with that new IP block. If there are problems, go back to ARIN and show them your work and if they can verify your work (or are simply lazy) you get a different block. ARIN puts the block into another quiet period. Maybe they use the work you did to clean up the block, maybe they don't. Cleaning up a block of IPs previously used by shady characters has a real cost, both in time and money. The argument as I see it is who bears the responsibility and cost of that cleanup. Beckman --- Peter Beckman Internet Guy beck...@angryox.com http://www.angryox.com/ ---
Re: Repeated Blacklisting / IP reputation
Skywing wrote: What's to stop spammers from doing this to cycle through blocks in rapid-fashion? This proposal seems easily abusable to me. Oh, I don't know, maybe ARIN staff can say no? The process is heavy with human interaction, there is nothing rapid about it, and bears no comparison to the automated process of registering a domain name. You'd know that if you ever had to make a request for a number resource from ARIN. ~Seth
Re: Repeated Blacklisting / IP reputation
On Wed, Sep 9, 2009 at 1:15 PM, Seth Mattinen se...@rollernet.us wrote: Skywing wrote: What's to stop spammers from doing this to cycle through blocks in rapid-fashion? This proposal seems easily abusable to me. Oh, I don't know, maybe ARIN staff can say no? The process is heavy with human interaction, there is nothing rapid about it, and bears no comparison to the automated process of registering a domain name. You'd know that if you ever had to make a request for a number resource from ARIN. The problem of tainted ipv4 allocations probably grows from here since at some point in the near future there isn't going to be much left in terms of clean space to allocate. We're running out of v4 addresses in case anyone forgot. Not sure that this is an ARIN problem more than an operational problem since RBL's are opt-in. An effort to identify RBL's that are behaving poorly is probably more interesting at this point, no? Best Regards, Marty ~Seth -- Martin Hannigan mar...@theicelandguy.com p: +16178216079 Power, Network, and Costs Consulting for Iceland Datacenters and Occupants
Re: Repeated Blacklisting / IP reputation
On Sep 8, 2009, at 5:20 PM, Joe Provo wrote: On Tue, Sep 08, 2009 at 01:43:39PM -0400, John Curran wrote: [snip] Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? RSS feed of whois churn? Tighter whois:irr coupling headed toward the ripe model such that irr-oriented tools can be applied to the problem? Joe - The RSS feed for as-issued blocks exists today, so RBL private list operators can practice good hygiene as desired: Announcement: https://www.arin.net/announcements/2009/20090622_daily_report.html Feed: http://lists.arin.net/pipermail/arin-issued/rss.xml Note that this is post-issuance, not as reclaimed/recovered because we do allow non-payment blocks to be recovered by coming current on payment, and thus it's not safe to presume that they're always issued to a new organization. With respect to moving towards tighter whois:IRR coupling, is there community desire for such in this region, and does that address this problem? e.g. Are blocks reissued in the RIPE region cleaner due to the tighter Whois:IRR linkage? Thanks! /John John Curran President and CEO ARIN
Re: Repeated Blacklisting / IP reputation
On Wed, 09 Sep 2009 15:13:44 EDT, Martin Hannigan said: Not sure that this is an ARIN problem more than an operational problem since RBL's are opt-in. An effort to identify RBL's that are behaving poorly is probably more interesting at this point, no? I suspect the problem isn't poor RBLs, it's all the little one-off block lists out there. The NANOG lurker in the next cubicle informs me that we currently carry an astounding 52,274 block entries (to be fair, a large portion is due to our vendor's somewhat-lacking block list - if we decide a /24 is bad, but then want to whitelist 1 IP, we have to de-aggregate to 254 black entries instead). We get maybe 5-6 blocked e-mail complaints a day - which *still* represents better performance for our end users than if we didn't carry around that many blocks (for comparison, we get at least 3-4 times that many tickets a day for people who forgot their e-mail password and need a reset). And yes, it's *very* intentional that we have a business process in place that makes it trivially easy for one of our users to open a I can't get e-mail from here and get it taken care of *very* quickly, but opening a We can't send e-mail to your users is a lot more challenging and time consuming (at least for the complaintant). Now, if we didn't have a dedicated, hard-working, and skeptical lurker in the next cubicle, our block list *would* be a mess.. ;) pgpIKBr5Pxz3V.pgp Description: PGP signature
Re: Repeated Blacklisting / IP reputation
Joe Greco wrote: John Curran wrote: On Sep 8, 2009, at 2:18 PM, JC Dill wrote: It seems simple and obvious that ARIN, RIPE, et. al. should determine the blacklist state of a reclaimed IP group and ensure that the IP group is usable before re-allocating it. When IPs are reclaimed, first check to see if the reclaimed IPs are on any readily checked RBL or private blacklist of major ISPs, corporations, universities, etc. If so, work with those groups to get the blocks removed *prior* to reissuing the IPs to a new entity. Before releasing the IPs to a new entity, double check that they are not being blocked (that any promises to remove them from a blacklist were actually fulfilled). Hold the IPs until you have determined that they aren't overly encumbered with prior blacklist blocks due to poor behavior of the previous entity. (The same should be done before allocating out of a new IP block, such as when you release the first set of IPs in a new /8.) In this case, it's not the RBL's that are the issue; the address block in question isn't on them. It's the ISP's and other firms using manual copies rather than actually following best practices. It's not that hard to make a list of the major ISPs, corporations, universities (entities with a large number of users), find willing contacts inside each organization (individual or role addresses you can email, and see if the email bounces, and who will reply if the email is received) and run some automated tests to see if the IPs are being blocked. In your follow-up email to me, you said you check dozens of RBLs - that is clearly insufficient - probably by an order of magnitude - of the entities you should check with. The number should be hundreds. A reasonably cluefull intern can provide you with a suitable list in short order, probably less than 1 day, and find suitable contacts inside each organization in a similar time frame - it might take a week total to build a list of ~500 entities and associated email addresses. Because of employee turn-over the list will need to be updated, ~1-10 old addresses purged and replaced with new ones on a monthly basis. Really? And you expect all these organizations to do ... what? Hire an intern to be permanent liaison to ARIN? I'm expecting ARIN to spend a few staff-hours (utilizing low-cost labor such as an intern) to setup the list for ARIN to use to check the status of returned IPs, and spend a few more staff hours setting up an automated system to utilize the list prior to releasing reclaimed IPs for reallocation. If, when using the list they discover out-dated addresses, spend a moment to find an updated address for that sole network. Most of this can easily be automated once setup - the only things that need to be dealt with by hand would be purging the list of outdated contacts and finding new ones, which shouldn't take much time since it's not a very large list, and many of the contacts would (over time) become role accounts that don't become outdated as often or as easily as personal accounts. Most of this is done by ARIN, not by the organizations they contact. All each organization has to do is permit one employee or role account to be used for IP block testing, and reply to test emails. The effort to setup a role account and autoresponder is minimal. Answer queries to whether or not IP space X is currently blocked (potentially at one of hundreds or thousands of points in their system, which corporate security may not wish to share, or even give some random intern access to)? Process reports of new ARIN delegations? What are you thinking they're going to do? And why should they care enough to do it? Because if they don't, they are needlessly blocking re-allocated IP addresses, potentially blocking their own users from receiving wanted email. Organizations could (and should) setup a role account and auto-responder for this purpose. Why isn't this being done now? Issuing reclaimed IPs is a lot like selling a used car, except that the buyer has no way to examine the state of the IPs you will issue them beforehand. Therefore it's up to you (ARIN, RIPE, et. al.) to ensure that they are just as good as any other IP block. It is shoddy business to take someone's money and then sneakily give them tainted (used) goods and expect them to deal with cleaning up the mess that the prior owner made, especially when you charge the same rate for untainted goods! Not applicable in this case, as noted above. What do you mean, not applicable? You take the money and issue IPs. There is no way for the buyer to know before hand if the IPs are tainted (used) or new. It is up to you (ARIN) to ensure that the goods (IPs) are suitable for the intended use. My analogy is entirely applicable, and I'm amazed you think otherwise. WOW. That's a hell of a statement. There is absolutely nothing that
Re: Repeated Blacklisting / IP reputation
JC Dill wrote: Joe Greco wrote: Answer queries to whether or not IP space X is currently blocked (potentially at one of hundreds or thousands of points in their system, which corporate security may not wish to share, or even give some random intern access to)? Process reports of new ARIN delegations? What are you thinking they're going to do? And why should they care enough to do it? Because if they don't, they are needlessly blocking re-allocated IP addresses, potentially blocking their own users from receiving wanted email. Organizations could (and should) setup a role account and auto-responder for this purpose. Perhaps they should, but until there is sufficient pain from their own users complaining about it there is no financial motivation to do so, and therefore many will not. I would guess that there are thousands of individual blocklists to this day blocking some of Sanford Wallace's and AGIS's old netblocks. As for a role account, there is postmaster. I would think that the best hope in the real world, rather than an autoresponder would be an RFC that clearly defines text accompanying an SMTP rejection notice triggered by a blocklist, detailing the blocklist and contact for removal. Perhaps encouraging those who code MTAs and DNSBL hooks into them to include such in the configuration files would be a good start. This still puts the onus on the sender or inheritor of the tainted netblock, but makes the search less painful and perhaps even somewhat able to be scripted. Note that this thread deals mostly with SMTP issues regarding DNSBLs, as those are the most common trouble point. We should also consider other forms of blocking/filtering of networks reclaimed from former virus/malware/DoS sources. -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: Repeated Blacklisting / IP reputation
On Sep 9, 2009, at 12:13 PM, Martin Hannigan wrote: The problem of tainted ipv4 allocations probably grows from here since at some point in the near future there isn't going to be much left in terms of clean space to allocate. We're running out of v4 addresses in case anyone forgot. Somewhat apropos to this discussion: http://blog.icann.org/2009/09/selecting-which-8-to-allocate-to-an-rir/ Regards, -drc
RE: Repeated Blacklisting / IP reputation
Along the same lines, I noticed that the worst Actor in recent memory (McColo - AS26780) stopped paying their bills to ARIN and their addresses have been returned to the pool. It's my opinion that a very select number of CIDR blocks (another example being the ones belonging to Cernel/InternetPath/Atrivo/etc, if it were ever fully extinguished) are, and forever will be, completely toxic and unusable to any legitimate enterprise. Arguments could be made that industry blacklists can and should be more flexible, but from the considerably more innocuous case in this thread, that is apparently not the modus operandi I'm curious to hear ARIN's thoughts, as well as the general NANOG populous, on whether you think it would be beneficial/possible to allocate the former blocks to $internetgoodguys (Shadowserver, Cymru, REN-ISAC, etc) for sinkholing and distribution of the data. /Many/ infected bots remain stranded post-McColo; large amounts of infection intelligence could easily be generated by such a move, and seemingly, would hurt no one. Although I'm in favor of revocation of allocations, similar to what happens in the DNS space for bad guys, this sort of move could obviously only happen if appropriate AUP sections were added into to the contracts (which I don't see happening). In the interm? This seems like a golden opportunity to gather some serious intel. Thoughts? Regards, Alex Lanstein From: John Curran [jcur...@arin.net] Sent: Tuesday, September 08, 2009 1:43 PM To: nanog@nanog.org Subject: Re: Repeated Blacklisting / IP reputation Folks - It appears that we have a real operational problem, in that ARIN does indeed reissue space that has been reclaimed/returned after a hold-down period, and but it appears that even once they are removed from the actual source RBL's, there are still ISP's who are manually updating these and hence block traffic much longer than necessary. I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? Thanks! /John John Curran President and CEO ARIN On Sep 8, 2009, at 11:16 AM, Ronald Cotoni wrote: Tom Pipes wrote: Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for lack of a better word) since we obtained it. It seems like every customer we have added has had repeated issues with being blacklisted by DUL and the cable carriers. (AOL, ATT, Charter, etc). I understand there is a process to getting removed, but it seems as if these IPs had been used and abused by the previous owner. We have done our best to ensure these blocks conform to RFC standards, including the proper use of reverse DNS pointers. I can resolve the issue very easily by moving these customers over to our other direct assigned 66.254.192.0/19 block. In the last year I have done this numerous times and have had no further issues with them. My question: Is there some way to clear the reputation of these blocks up, or start over to prevent the amount of time we are spending with each customer troubleshooting unnecessary RBL and reputation blacklisting? I have used every opportunity to use the automated removal links from the SMTP rejections, and worked with the RBL operators directly. Most of what I get are cynical responses and promises that it will be fixed. If there is any question, we perform inbound and outbound scanning of all e-mail, even though we know that this appears to be something more relating to the block itself. Does anyone have any suggestions as to how we can clear this issue up? Comments on or off list welcome. Thanks, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com Unfortunately, there is no real good way to get yourself completely delisted. We are experiencing that with a /18 we got from ARIN recently and it is basically the RBL's not updating or perhaps they are not checking the ownership of the ip's as compared to before. On some RBL's, we have IP addresses that have been listed since before the company I work for even existed. Amazing right? -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Re: Repeated Blacklisting / IP reputation
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, Sep 9, 2009 at 7:18 PM, Alex Lanstein alanst...@fireeye.com wrote: Along the same lines, I noticed that the worst Actor in recent memory (McColo - AS26780) stopped paying their bills to ARIN and their addresses have been returned to the pool. It's my opinion that a very select number of CIDR blocks (another example being the ones belonging to Cernel/InternetPath/Atrivo/etc, if it were ever fully extinguished) are, and forever will be, completely toxic and unusable to any legitimate enterprise. Arguments could be made that industry blacklists can and should be more flexible, but from the considerably more innocuous case in this thread, that is apparently not the modus operandi With regards to Cernel/Internet Path/UkrTelGrp, it needs to be extinguished first. :-) - - ferg -BEGIN PGP SIGNATURE- Version: PGP Desktop 9.5.3 (Build 5003) wj8DBQFKqGZIq1pz9mNUZTMRAnE3AKCL76mNabIzAf5FCWRfqci3YW5QKACgtLNJ AXSIGuT1tIe0R+tm+VL/Flc= =NYQS -END PGP SIGNATURE- -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawgster(at)gmail.com ferg's tech blog: http://fergdawg.blogspot.com/
Re: Repeated Blacklisting / IP reputation
On Sep 9, 2009, at 7:18 PM, Alex Lanstein wrote: Along the same lines, I noticed that the worst Actor in recent memory (McColo - AS26780) stopped paying their bills to ARIN and their addresses have been returned to the pool. It's my opinion that a very select number of CIDR blocks (another example being the ones belonging to Cernel/InternetPath/Atrivo/etc, if it were ever fully extinguished) are, and forever will be, completely toxic and unusable to any legitimate enterprise. Arguments could be made that industry blacklists can and should be more flexible, but from the considerably more innocuous case in this thread, that is apparently not the modus operandi Putting these addresses back into use does not mean that they have to be allocated to networks where they'll number mail servers. ARIN staff is doubtless aware of the history of these blocks and will presumably do their best to allocate them to networks that aren't intended to host mail servers. Regards, Leo
Re: Repeated Blacklisting / IP reputation
On Wed, Sep 9, 2009 at 11:30 PM, Leo Vegoda leo.veg...@icann.org wrote: On Sep 9, 2009, at 7:18 PM, Alex Lanstein wrote: Along the same lines, I noticed that the worst Actor in recent memory (McColo - AS26780) stopped paying their bills to ARIN and their addresses have been returned to the pool. It's my opinion that a very select number of CIDR blocks (another example being the ones belonging to Cernel/InternetPath/Atrivo/etc, if it were ever fully extinguished) are, and forever will be, completely toxic and unusable to any legitimate enterprise. Arguments could be made that industry blacklists can and should be more flexible, but from the considerably more innocuous case in this thread, that is apparently not the modus operandi Putting these addresses back into use does not mean that they have to be allocated to networks where they'll number mail servers. ARIN staff is doubtless aware of the history of these blocks and will presumably do their best to allocate them to networks that aren't intended to host mail servers. Regards, Leo Not sure when ICANN got into the business of economic bailouts, but the mechanism that ICANN has defined seems patently unfair. Determining who is worthy of allocations based on a class without community input into a policy debate is bad. ObOps: Chasing down all of this grunge ain't cheap or fair. Best, Martin -- Martin Hannigan mar...@theicelandguy.com p: +16178216079 Power, Network, and Costs Consulting for Iceland Datacenters and Occupants
Re: Repeated Blacklisting / IP reputation
In message e1decfc9-80ef-40fa-9d98-5c622aacc...@icann.org, Leo Vegoda writes: On Sep 9, 2009, at 7:18 PM, Alex Lanstein wrote: Along the same lines, I noticed that the worst Actor in recent =20 memory (McColo - AS26780) stopped paying their bills to ARIN and =20 their addresses have been returned to the pool. It's my opinion that a very select number of CIDR blocks (another =20 example being the ones belonging to Cernel/InternetPath/Atrivo/etc, =20 if it were ever fully extinguished) are, and forever will be, =20 completely toxic and unusable to any legitimate enterprise. =20 Arguments could be made that industry blacklists can and should be =20 more flexible, but from the considerably more innocuous case in this =20 thread, that is apparently not the modus operandi Putting these addresses back into use does not mean that they have to =20 be allocated to networks where they'll number mail servers. ARIN staff =20 is doubtless aware of the history of these blocks and will presumably =20 do their best to allocate them to networks that aren't intended to =20 host mail servers. Regards, Leo What a load of rubbish. How is ARIN or any RIR/LIR supposed to know the intent of use? Push has come to shove and those that have incorrectly treated address assignment as immutable will need to correct their ways (excluding legacy assignments). This will be painful for some. Note we all could start using IPv6 and avoid this problem altogether. There is nothing stopping us using IPv6 especially for MTA's. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Re: Repeated Blacklisting / IP reputation
Tom Pipes wrote: Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for lack of a better word) since we obtained it. It seems like every customer we have added has had repeated issues with being blacklisted by DUL and the cable carriers. (AOL, ATT, Charter, etc). I understand there is a process to getting removed, but it seems as if these IPs had been used and abused by the previous owner. We have done our best to ensure these blocks conform to RFC standards, including the proper use of reverse DNS pointers. I can resolve the issue very easily by moving these customers over to our other direct assigned 66.254.192.0/19 block. In the last year I have done this numerous times and have had no further issues with them. My question: Is there some way to clear the reputation of these blocks up, or start over to prevent the amount of time we are spending with each customer troubleshooting unnecessary RBL and reputation blacklisting? I have used every opportunity to use the automated removal links from the SMTP rejections, and worked with the RBL operators directly. Most of what I get are cynical responses and promises that it will be fixed. If there is any question, we perform inbound and outbound scanning of all e-mail, even though we know that this appears to be something more relating to the block itself. Does anyone have any suggestions as to how we can clear this issue up? Comments on or off list welcome. Thanks, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com Unfortunately, there is no real good way to get yourself completely delisted. We are experiencing that with a /18 we got from ARIN recently and it is basically the RBL's not updating or perhaps they are not checking the ownership of the ip's as compared to before. On some RBL's, we have IP addresses that have been listed since before the company I work for even existed. Amazing right?
Re: Repeated Blacklisting / IP reputation
Folks - It appears that we have a real operational problem, in that ARIN does indeed reissue space that has been reclaimed/returned after a hold-down period, and but it appears that even once they are removed from the actual source RBL's, there are still ISP's who are manually updating these and hence block traffic much longer than necessary. I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? Thanks! /John John Curran President and CEO ARIN On Sep 8, 2009, at 11:16 AM, Ronald Cotoni wrote: Tom Pipes wrote: Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for lack of a better word) since we obtained it. It seems like every customer we have added has had repeated issues with being blacklisted by DUL and the cable carriers. (AOL, ATT, Charter, etc). I understand there is a process to getting removed, but it seems as if these IPs had been used and abused by the previous owner. We have done our best to ensure these blocks conform to RFC standards, including the proper use of reverse DNS pointers. I can resolve the issue very easily by moving these customers over to our other direct assigned 66.254.192.0/19 block. In the last year I have done this numerous times and have had no further issues with them. My question: Is there some way to clear the reputation of these blocks up, or start over to prevent the amount of time we are spending with each customer troubleshooting unnecessary RBL and reputation blacklisting? I have used every opportunity to use the automated removal links from the SMTP rejections, and worked with the RBL operators directly. Most of what I get are cynical responses and promises that it will be fixed. If there is any question, we perform inbound and outbound scanning of all e-mail, even though we know that this appears to be something more relating to the block itself. Does anyone have any suggestions as to how we can clear this issue up? Comments on or off list welcome. Thanks, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com Unfortunately, there is no real good way to get yourself completely delisted. We are experiencing that with a /18 we got from ARIN recently and it is basically the RBL's not updating or perhaps they are not checking the ownership of the ip's as compared to before. On some RBL's, we have IP addresses that have been listed since before the company I work for even existed. Amazing right?
Re: Repeated Blacklisting / IP reputation
John, its about the same situation you get when people use manually updated bogon filters. A much larger problem, I must admit .. having ISPs follow the maawg best practices might help, that - and attending MAAWG sessions (www.maawg.org - Published Documents) That said most of the larger players already attend MAAWG - that leaves rural ISPs, small universities, corporate mailservers etc etc that dont have full time postmasters, and where you're more likely to run into this issue. If you see actual large carriers with outdated blocklist entries after they're removed from (say) the spamhaus pbl, then maybe MAAWG needs to come to nanog / arin meetings .. plenty of maawg types attend those that all that needs to be done is to free up a presentation slot or two. --srs On Tue, Sep 8, 2009 at 11:13 PM, John Curranjcur...@arin.net wrote: Folks - It appears that we have a real operational problem, in that ARIN does indeed reissue space that has been reclaimed/returned after a hold-down period, and but it appears that even once they are removed from the actual source RBL's, there are still ISP's who are manually updating these and hence block traffic much longer than necessary. I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? Thanks! /John John Curran President and CEO ARIN On Sep 8, 2009, at 11:16 AM, Ronald Cotoni wrote: Tom Pipes wrote: Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for lack of a better word) since we obtained it. It seems like every customer we have added has had repeated issues with being blacklisted by DUL and the cable carriers. (AOL, ATT, Charter, etc). I understand there is a process to getting removed, but it seems as if these IPs had been used and abused by the previous owner. We have done our best to ensure these blocks conform to RFC standards, including the proper use of reverse DNS pointers. I can resolve the issue very easily by moving these customers over to our other direct assigned 66.254.192.0/19 block. In the last year I have done this numerous times and have had no further issues with them. My question: Is there some way to clear the reputation of these blocks up, or start over to prevent the amount of time we are spending with each customer troubleshooting unnecessary RBL and reputation blacklisting? I have used every opportunity to use the automated removal links from the SMTP rejections, and worked with the RBL operators directly. Most of what I get are cynical responses and promises that it will be fixed. If there is any question, we perform inbound and outbound scanning of all e-mail, even though we know that this appears to be something more relating to the block itself. Does anyone have any suggestions as to how we can clear this issue up? Comments on or off list welcome. Thanks, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com Unfortunately, there is no real good way to get yourself completely delisted. We are experiencing that with a /18 we got from ARIN recently and it is basically the RBL's not updating or perhaps they are not checking the ownership of the ip's as compared to before. On some RBL's, we have IP addresses that have been listed since before the company I work for even existed. Amazing right? -- Suresh Ramasubramanian (ops.li...@gmail.com)
Re: Repeated Blacklisting / IP reputation
Suresh Ramasubramanian wrote: That said most of the larger players already attend MAAWG - that leaves rural ISPs, small universities, corporate mailservers etc etc that dont have full time postmasters, and where you're more likely to run into this issue. I've found the opposite to hold true more often. Smaller organizations can use public blacklists for free, due to their low volume, and so have little incentive to run their own local blacklist. I've typically seen the larger organizations run their own blacklists and are much more difficult to contact for removal.
Re: Repeated Blacklisting / IP reputation
Suresh Ramasubramanian wrote: John, its about the same situation you get when people use manually updated bogon filters. A much larger problem, I must admit .. having ISPs follow the maawg best practices might help, that - and attending MAAWG sessions (www.maawg.org - Published Documents) That said most of the larger players already attend MAAWG - that leaves rural ISPs, small universities, corporate mailservers etc etc that dont have full time postmasters, and where you're more likely to run into this issue. I was always under the impression that smaller orgs were not allowed to join the MAAWG club. ~Seth
Re: Repeated Blacklisting / IP reputation
John Curran wrote: Folks - It appears that we have a real operational problem, in that ARIN does indeed reissue space that has been reclaimed/returned after a hold-down period, and but it appears that even once they are removed from the actual source RBL's, there are still ISP's who are manually updating these and hence block traffic much longer than necessary. I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? I don't think there is an excellent reason, more likely inertia and no real incentive to put forth the effort to proactively remove addresses. Many ISPs and organizations have their own private blocklists not associated with the widely known DNSBLs. Typically during or immediately after a spam run the mail administrator will manually add offending addresses or netblocks. Spamtrap hits may do this automatically. There isn't any real incentive for people to go back and remove addresses unless they're notified by their own customers that legitimate mail coming from those addresses is being blocked. Because these blocklists are individually maintained, there is no central registry or means to clean them up when an IP assignment changes. To make matters worse, some organizations may simply ACL the IP space so that the TCP connection is never made in the first place (bad, looks like a network problem rather than deliberate filtering), some may drop it during SMTP with no clear indication as to the reason (less bad, as there is at least a hint that it could be filtering), and some may actually accept the mail and then silently discard it (worst). In addition there are several DNSBLs with different policies regarding delisting. Some just time out after a period of time since abuse was detected. Some require action in the form of a delisting request. Some require a delisting request and a time period with no abuse. Some (the old SPEWS list) may not be easily reached or have well defined policies. In meatspace, once a neighborhood winds up with a reputation of being rife with drive-by shootings, gang activity and drug dealing it may take a long time after the last of the graffiti is gone before some cab drivers will go there. -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: Repeated Blacklisting / IP reputation
On Tue, 8 Sep 2009, John Curran wrote: I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? Most small to midsize networks probably have a block it and forget it policy. The facts that the spammer moved on, the IPs eventually got returned to the RIR and reallocated to a different network go unnoticed until the new LIR/ISP notifies those blocking the addresses that the addresses have changed hands. Ideally, the network doing the blocking will know when they started blocking an IP, look at whois, and agree that the block no longer makes sense. I'm sure some will have no idea when or why they started blocking an IP, and might be reluctant to unblock it. This assumes you can actually get in touch with someone with the access and understanding of the issues to have a conversation about their blocking. Some networks make that nearly impossible. I ran into such situations early on when trying to contact networks about their outdated bogon filters when Atlantic.net got a slice of 69/8. This blocking (or variations of it) has been a problem for about a decade. http://www.michnet.net/mail.archives/nanog/2001-08/msg00448.html I don't think there is any blanket solution to this issue. Too many of the networks doing the blocking likely don't participate in any forum where the RIRs will be reach people who care and can do something. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: Repeated Blacklisting / IP reputation
On Sep 8, 2009, at 11:13 AM, Jay Hennigan wrote: John Curran wrote: snip I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? I don't think there is an excellent reason, more likely inertia and no real incentive to put forth the effort to proactively remove addresses. snip In addition there are several DNSBLs with different policies regarding delisting. Some just time out after a period of time since abuse was detected. Some require action in the form of a delisting request. Some require a delisting request and a time period with no abuse. Some (the old SPEWS list) may not be easily reached or have well defined policies. In meatspace, once a neighborhood winds up with a reputation of being rife with drive-by shootings, gang activity and drug dealing it may take a long time after the last of the graffiti is gone before some cab drivers will go there. -- Jay Hennigan - CCIE #7880 snip I think this most accurately reflects the reality I see dealing with mostly enterprises and mid-to-large xSPs. A lot of mid-range enterprises out there have legacy free (often meaning subscriptions aren't enforced) DNSBLs in place that were configured years ago as a desperate attempt to reduce e-mail load, before there were well-maintained alternatives. The problem is that these services usually don't have the resources to put a lot of advanced automation and sophisticated logic into place, so delisting is a huge hassle (and some times resembles extortion). There are some quality free services, such as Spamhaus (speaking personally), but they're few and far between. I've had better luck convincing customers (or customers of customers) to stop using the poorly-maintained legacy DNSBLs than I've had getting customers delisted from such services. YMMV. Brian Keefer Sr. Solutions Architect Defend email. Protect data.
Re: Repeated Blacklisting / IP reputation
On Tue, 08 Sep 2009 13:43:39 EDT, John Curran said: I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... If I'm a smaller shop with limited clue, there's 3 likely colloraries: 1) Even a smallish spam blast is big enough to cause me operational difficulties, so I'm tempted to throw in a block to fix it. 2) Once the spammers have moved on, it's unlikely that I have enough customers trying to reach the blocked address space and complaining for me to fix it, and the people *in* that address space can't successfully complain because I've blocked it. 3) The damage to traffic is of consequence to the remote site, but isn't a revenue-impacting issue for *ME*. The third point is the biggie here. pgpSZgeKu8pfq.pgp Description: PGP signature
Re: Repeated Blacklisting / IP reputation
On Tue, Sep 08, 2009 at 10:16:33AM -0500, Ronald Cotoni wrote: Tom Pipes wrote: Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for lack of a better word) since we obtained it. It seems like every customer we have added has had repeated issues with being blacklisted by DUL and the cable carriers. (AOL, ATT, Charter, etc). I understand there is a process to getting removed, but it seems as if these IPs had been used and abused by the previous owner. We have done our best to ensure these blocks conform to RFC standards, including the proper use of reverse DNS pointers. I can resolve the issue very easily by moving these customers over to our other direct assigned 66.254.192.0/19 block. In the last year I have done this numerous times and have had no further issues with them. My question: Is there some way to clear the reputation of these blocks up, or start over to prevent the amount of time we are spending with each customer troubleshooting unnecessary RBL and reputation blacklisting? I have used every opportunity to use the automated removal links from the SMTP rejections, and worked with the RBL operators directly. Most of what I get are cynical responses and promises that it will be fixed. If there is any question, we perform inbound and outbound scanning of all e-mail, even though we know that this appears to be something more relating to the block itself. Does anyone have any suggestions as to how we can clear this issue up? Comments on or off list welcome. Thanks, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com Unfortunately, there is no real good way to get yourself completely delisted. We are experiencing that with a /18 we got from ARIN recently and it is basically the RBL's not updating or perhaps they are not checking the ownership of the ip's as compared to before. On some RBL's, we have IP addresses that have been listed since before the company I work for even existed. Amazing right? This is not actually a new problem. ISPs have been fighting this for some time. When a dud customer spams from a given IP range and gets it placed in various RBLs, when that customer is booted or otherwise removed, that block will probably get reissued. The new customer then calls up and says, my email isn't getting through. All it takes is a little investigation and the cause becomes clear. In my experience, there is absolutely no way to deal with this other than contacting the companies your customer is trying to email one by one. Not all of them will respond to you but when they are slow or do not act at all, quite often if the recipient on the other end calls them up and says, WTF? it generates more action. Sadly, I do not foresee this problem getting any easier. Best practices for the public or subscription RBLs should be to place a TTL on the entry of no more than, say, 90 days or thereabouts. Best practices for manual entry should be to either keep a list of what and when or periodically to simply blow the whole list away and start anew to get rid of stale entries. Of course, that is probably an unreal expectation. -Wayne --- Wayne Bouchard w...@typo.org Network Dude http://www.typo.org/~web/
Re: Repeated Blacklisting / IP reputation
On Tue, 8 Sep 2009, John Curran wrote: I'm sure there's an excellent reason why these addresses stay blocked, but am unable to fathom what exactly that is... Could some folks from the appropriate networks explain why this is such a problem and/or suggest additional steps that ARIN or the receipts should be taking to avoid this situation? Most small to midsize networks probably have a block it and forget it policy. The facts that the spammer moved on, the IPs eventually got returned to the RIR and reallocated to a different network go unnoticed until the new LIR/ISP notifies those blocking the addresses that the addresses have changed hands. Ideally, the network doing the blocking will know when they started blocking an IP, look at whois, and agree that the block no longer makes sense. I'm sure some will have no idea when or why they started blocking an IP, and might be reluctant to unblock it. This assumes you can actually get in touch with someone with the access and understanding of the issues to have a conversation about their blocking. Some networks make that nearly impossible. I ran into such situations early on when trying to contact networks about their outdated bogon filters when Atlantic.net got a slice of 69/8. This blocking (or variations of it) has been a problem for about a decade. http://www.michnet.net/mail.archives/nanog/2001-08/msg00448.html I don't think there is any blanket solution to this issue. Too many of the networks doing the blocking likely don't participate in any forum where the RIRs will be reach people who care and can do something. It should be pretty clear that reused IP space is going to represent a problem. There is no mechanism for LIR/ISP notif[cation to] those blocking the addresses that the addresses have changed hands. Even if there were, this would be subject to potential gaming by spammers, such as SWIP of a block to SpammerXCo, followed by an automatic unblock when the ISP unSWIP's it and SWIP's it to EmailBlasterB - of course, the same company. How do we manage this into the future? IPv6 shows some promise in terms of delegation of larger spaces, which could in turn suggest that reuse policies that discourage rapid reuse would be a best practice. However, that is more or less just acknowledging the status quo; networks are likely to continue blocking for various reasons and for random periods. A remote site being unable to communicate with us is not particularly important except to the extent that it ends up distressing users here; however, for larger sites, the blocked list could end up being significant. It seems like it *could* be useful to have a system to notify of network delegation changes, but it also seems like if this was particularly important to anyone, then someone would have found a trivial way to implement at least a poor man's version of it. For example, record the ASN of a blocked IP address and remove the block when the ASN changes... ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
On Tue, 8 Sep 2009, Joe Greco wrote: It seems like it *could* be useful to have a system to notify of network delegation changes, but it also seems like if this was particularly important to anyone, then someone would have found a trivial way to implement at least a poor man's version of it. For example, record the ASN of a blocked IP address and remove the block when the ASN changes... That too, would be easily gamed by spammers. Just get multiple ASN's and bounce your dirty IPs around between them to clean them. The IP space being a direct (RIR-LIR) allocation having been made after the blocking was initiated is a pretty clear sign that the space has actually changed hands, and seems like it would be fairly difficult (if at all possible) to game. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: Repeated Blacklisting / IP reputation
On Tue, 8 Sep 2009, Wayne E. Bouchard wrote: This is not actually a new problem. ISPs have been fighting this for some time. When a dud customer spams from a given IP range and gets it placed in various RBLs, when that customer is booted or otherwise removed, that block will probably get reissued. The new customer then calls up and says, my email isn't getting through. All it takes is a The difference/issue here is that it's easy for you when turning down or turning up a customer to check the IP space being revoked/assigned in the various popular public DNSBLs, sparing your customers the headache of being assigned blacklisted IPs. Until your next customer starts using the space and can't send us email, you have no way of knowing that we null routed the subnet on our MX cluster. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: Repeated Blacklisting / IP reputation
Seth Mattinen wrote: I was always under the impression that smaller orgs were not allowed to join the MAAWG club. They're allowed. At $4k/year minimum, up to $25K/year. By the way, among the members... Experian CheetahMail ExactTarget, Inc Responsys, Inc. Vertical Response, Inc Yesmail -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: Repeated Blacklisting / IP reputation
On Tue, 8 Sep 2009, Joe Greco wrote: It seems like it *could* be useful to have a system to notify of network delegation changes, but it also seems like if this was particularly important to anyone, then someone would have found a trivial way to implement at least a poor man's version of it. For example, record the ASN of a blocked IP address and remove the block when the ASN changes... That too, would be easily gamed by spammers. Just get multiple ASN's and bounce your dirty IPs around between them to clean them. The IP space being a direct (RIR-LIR) allocation having been made after the blocking was initiated is a pretty clear sign that the space has actually changed hands, and seems like it would be fairly difficult (if at all possible) to game. Right, but they'll only do that if they're aware of such a system and it is significant enough to make a dent in them. Further, it would be a mistake to assume that *just* changing ASN's would be sufficient. The act of changing ASN's could act as a trigger to re-whois ARIN for an update of ownership, for example. The fact is that the information to trigger a re-query of ownership upon a redelegation sort-of already exists, though it is clearly imperfect. My point was that if it was actually useful to notice when an IP delegation moved, someone would already have made up a system to do this somehow. So my best guess is that there isn't a really strong incentive to pursue some sort of notification system, because you could pretty much do it as it stands. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
John Curran wrote: On Sep 8, 2009, at 2:18 PM, JC Dill wrote: It seems simple and obvious that ARIN, RIPE, et. al. should determine the blacklist state of a reclaimed IP group and ensure that the IP group is usable before re-allocating it. When IPs are reclaimed, first check to see if the reclaimed IPs are on any readily checked RBL or private blacklist of major ISPs, corporations, universities, etc. If so, work with those groups to get the blocks removed *prior* to reissuing the IPs to a new entity. Before releasing the IPs to a new entity, double check that they are not being blocked (that any promises to remove them from a blacklist were actually fulfilled). Hold the IPs until you have determined that they aren't overly encumbered with prior blacklist blocks due to poor behavior of the previous entity. (The same should be done before allocating out of a new IP block, such as when you release the first set of IPs in a new /8.) In this case, it's not the RBL's that are the issue; the address block in question isn't on them. It's the ISP's and other firms using manual copies rather than actually following best practices. It's not that hard to make a list of the major ISPs, corporations, universities (entities with a large number of users), find willing contacts inside each organization (individual or role addresses you can email, and see if the email bounces, and who will reply if the email is received) and run some automated tests to see if the IPs are being blocked. In your follow-up email to me, you said you check dozens of RBLs - that is clearly insufficient - probably by an order of magnitude - of the entities you should check with. The number should be hundreds. A reasonably cluefull intern can provide you with a suitable list in short order, probably less than 1 day, and find suitable contacts inside each organization in a similar time frame - it might take a week total to build a list of ~500 entities and associated email addresses. Because of employee turn-over the list will need to be updated, ~1-10 old addresses purged and replaced with new ones on a monthly basis. Why isn't this being done now? Issuing reclaimed IPs is a lot like selling a used car, except that the buyer has no way to examine the state of the IPs you will issue them beforehand. Therefore it's up to you (ARIN, RIPE, et. al.) to ensure that they are just as good as any other IP block. It is shoddy business to take someone's money and then sneakily give them tainted (used) goods and expect them to deal with cleaning up the mess that the prior owner made, especially when you charge the same rate for untainted goods! Not applicable in this case, as noted above. What do you mean, not applicable? You take the money and issue IPs. There is no way for the buyer to know before hand if the IPs are tainted (used) or new. It is up to you (ARIN) to ensure that the goods (IPs) are suitable for the intended use. My analogy is entirely applicable, and I'm amazed you think otherwise. So, back to the question: could someone explain why they've got copies of the RBL's in their network which don't get updated on any reasonable refresh interval? (weekly? monthly?) The why really isn't at issue - it happens and it's going to keep happening. The question is what are you (ARIN) going to do about it? Give me the serenity to accept the things I cannot change, The courage to change the things I can, And the wisdom to know the difference. You (ARIN et. al.) don't have any ability to change the why. What you can change is how you go about determining if an IP block is suitable for reallocation or not, and what steps you take to repair IP blocks that aren't suitable for reallocation. jc - posted to NANOG since John indicated that he thought his reply to me was going to NANOG as well.
Re: Repeated Blacklisting / IP reputation
On Tue, Sep 08, 2009 at 02:34:10PM -0500, Joe Greco wrote: there is a fundamental disconnect here. the IP space is neutral. it has no bias toward or against social behaviours. its a tool. the actual/real target here are the people who are using these tools to be antisocial. blacklisting IP space is always reactive and should only beused in emergency and as a -TEMPORARY- expedient. IMHO of course., YMMV. Show me ONE major MTA which allows you to configure an expiration for an ACL entry. call me old skool... VI works a treat and I'm told there is this thing called emacs ... but i remain dubious. The problem with your opinion, and it's a fine opinion, and it's even a good opinion, is that it has very little relationship to the tools which are given to people in order to accomplish blocking. Kind of the question I was contemplating in my other message of minutes ago. if all you have is a hammer... folks need better tools. If people were given an option to block this IP for 30 minutes, 24 hours, 30 days, 12 months, 5 years, or forever - I wonder how many people would just shrug and click forever. which is their choice. please show me the mandate for accepting routes/packets from any/everywhere? me, i'd want the option to block 192.0.2.0/24 as long as it is announced by AS 0 and the whois data points to RIAA as the registered contact e.g. not just a temporal block. or - if traffic from 192.0.2.80 increases more than 65% in a 150 second interval, block the IP for 27 minutes. or - allow any/all traffic from 192.0.2.42 - regardless of the blocking on 192.0.2.0/24 the mind boggles. This may lead to the discovery of another fundamental disconnect - or two. such is the course of human nature. Sigh. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
Jason Bertoch wrote: Suresh Ramasubramanian wrote: That said most of the larger players already attend MAAWG - that leaves rural ISPs, small universities, corporate mailservers etc etc that dont have full time postmasters, and where you're more likely to run into this issue. I've found the opposite to hold true more often. Smaller organizations can use public blacklists for free, due to their low volume, and so have little incentive to run their own local blacklist. I've typically seen the larger organizations run their own blacklists and are much more difficult to contact for removal. Take for example GoDaddy's hosted email service. They are using a local, outdated copy of SORBS that has one of my personal servers listed in it. It was an open proxy for about week nearly 3 years ago and still they have it listed. The upside is that I've demonstrated GoDaddy's email incompetence to potential customers and gotten them to switch to our own mail services. Their loss, my gain. Justin
Re: Repeated Blacklisting / IP reputation
John Curran wrote: On Sep 8, 2009, at 2:18 PM, JC Dill wrote: It seems simple and obvious that ARIN, RIPE, et. al. should determine the blacklist state of a reclaimed IP group and ensure that the IP group is usable before re-allocating it. When IPs are reclaimed, first check to see if the reclaimed IPs are on any readily checked RBL or private blacklist of major ISPs, corporations, universities, etc. If so, work with those groups to get the blocks removed *prior* to reissuing the IPs to a new entity. Before releasing the IPs to a new entity, double check that they are not being blocked (that any promises to remove them from a blacklist were actually fulfilled). Hold the IPs until you have determined that they aren't overly encumbered with prior blacklist blocks due to poor behavior of the previous entity. (The same should be done before allocating out of a new IP block, such as when you release the first set of IPs in a new /8.) In this case, it's not the RBL's that are the issue; the address block in question isn't on them. It's the ISP's and other firms using manual copies rather than actually following best practices. It's not that hard to make a list of the major ISPs, corporations, universities (entities with a large number of users), find willing contacts inside each organization (individual or role addresses you can email, and see if the email bounces, and who will reply if the email is received) and run some automated tests to see if the IPs are being blocked. In your follow-up email to me, you said you check dozens of RBLs - that is clearly insufficient - probably by an order of magnitude - of the entities you should check with. The number should be hundreds. A reasonably cluefull intern can provide you with a suitable list in short order, probably less than 1 day, and find suitable contacts inside each organization in a similar time frame - it might take a week total to build a list of ~500 entities and associated email addresses. Because of employee turn-over the list will need to be updated, ~1-10 old addresses purged and replaced with new ones on a monthly basis. Really? And you expect all these organizations to do ... what? Hire an intern to be permanent liaison to ARIN? Answer queries to whether or not IP space X is currently blocked (potentially at one of hundreds or thousands of points in their system, which corporate security may not wish to share, or even give some random intern access to)? Process reports of new ARIN delegations? What are you thinking they're going to do? And why should they care enough to do it? Why isn't this being done now? Issuing reclaimed IPs is a lot like selling a used car, except that the buyer has no way to examine the state of the IPs you will issue them beforehand. Therefore it's up to you (ARIN, RIPE, et. al.) to ensure that they are just as good as any other IP block. It is shoddy business to take someone's money and then sneakily give them tainted (used) goods and expect them to deal with cleaning up the mess that the prior owner made, especially when you charge the same rate for untainted goods! Not applicable in this case, as noted above. What do you mean, not applicable? You take the money and issue IPs. There is no way for the buyer to know before hand if the IPs are tainted (used) or new. It is up to you (ARIN) to ensure that the goods (IPs) are suitable for the intended use. My analogy is entirely applicable, and I'm amazed you think otherwise. WOW. That's a hell of a statement. There is absolutely nothing that ARIN can do if I decide I'm going to have our servers block connections from networks ending in an odd bit. Nobody is in a position to ensure that ANY Internet connection or IP space is suitable for the intended use. Welcome to the Internet. So, back to the question: could someone explain why they've got copies of the RBL's in their network which don't get updated on any reasonable refresh interval? (weekly? monthly?) The why really isn't at issue - it happens and it's going to keep happening. The question is what are you (ARIN) going to do about it? Give me the serenity to accept the things I cannot change, The courage to change the things I can, And the wisdom to know the difference. You (ARIN et. al.) don't have any ability to change the why. What you can change is how you go about determining if an IP block is suitable for reallocation or not, and what steps you take to repair IP blocks that aren't suitable for reallocation. So, in addition to just registering IP space, it's also their job to clean it up? I'm sorry, I agree that there's a problem, but this just sounds like it isn't feasible. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one
Re: Repeated Blacklisting / IP reputation
Seth Mattinen wrote: I was always under the impression that smaller orgs were not allowed to join the MAAWG club. I've heard that, too, but have no idea where it comes from. It's not true; there's no size requirement or anything like that. http://www.maawg.org/ has the membership application and other info. -- J.D. Falk Co-Chair, Program Committee Messaging Anti-Abuse Working Group
Re: Repeated Blacklisting / IP reputation
J.D. Falk wrote: Seth Mattinen wrote: I was always under the impression that smaller orgs were not allowed to join the MAAWG club. I've heard that, too, but have no idea where it comes from. It's not true; there's no size requirement or anything like that. http://www.maawg.org/ has the membership application and other info. The $4000/year minimum membership fee is a non-starter for small organizations who are already strapped for operating cash as it is. This is probably where the perception comes from. -- William Astle l...@l-w.ca
Re: Repeated Blacklisting / IP reputation
there is a fundamental disconnect here. the IP space is neutral. it has no bias toward or against social behaviours. its a tool. the actual/real target here are the people who are using these tools to be antisocial. blacklisting IP space is always reactive and should only beused in emergency and as a -TEMPORARY- expedient. IMHO of course., YMMV. Show me ONE major MTA which allows you to configure an expiration for an ACL entry. The problem with your opinion, and it's a fine opinion, and it's even a good opinion, is that it has very little relationship to the tools which are given to people in order to accomplish blocking. Kind of the question I was contemplating in my other message of minutes ago. If people were given an option to block this IP for 30 minutes, 24 hours, 30 days, 12 months, 5 years, or forever - I wonder how many people would just shrug and click forever. This may lead to the discovery of another fundamental disconnect - or two. Sigh. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Repeated Blacklisting / IP reputation
MAAWG is has no size limitations as to members. Yes we do have a $4000 supporter membership. This has not proved a barrier to many organisations. Mike O'Reirdan Chairman, MAAWG - Original Message - From: Benjamin Billon bbillon...@splio.fr To: nanog@nanog.org nanog@nanog.org Sent: Tue Sep 08 17:17:58 2009 Subject: Re: Repeated Blacklisting / IP reputation ISPs can be invited and there are specific meetings for them (closed to other members). There're also whitepapers for ISP (and others). But I agree, hoping ALL the ISPs join MAAWG or even hear about it is utopian. -- Benjamin William Astle a écrit : J.D. Falk wrote: Seth Mattinen wrote: I was always under the impression that smaller orgs were not allowed to join the MAAWG club. I've heard that, too, but have no idea where it comes from. It's not true; there's no size requirement or anything like that. http://www.maawg.org/ has the membership application and other info. The $4000/year minimum membership fee is a non-starter for small organizations who are already strapped for operating cash as it is. This is probably where the perception comes from.
Re: Repeated Blacklisting / IP reputation
Joe Greco wrote: I'm sorry, I agree that there's a problem, but this just sounds like it isn't feasible. Some people suffer from the culturally ingrained inability to understand that certain kinds of problems just can't. Be. Solved. And/or they aren't worth solving under present circumstances. -- Alex Balashov - Principal Evariste Systems Web : http://www.evaristesys.com/ Tel : (+1) (678) 954-0670 Direct : (+1) (678) 954-0671
Re: Repeated Blacklisting / IP reputation
I am amazed with the amount of thoughtful comments I have seen, both on and off list. It really illustrates that people are willing to try to help out, but there is an overall lack of clear direction on how to improve things. Most of us seem to adopt that which has always just worked for us. Don't get me wrong, I'm sure there are a lot of improvements/mods going on with RBL operators in terms of the technology and how they choose who to block. I'm also certain that most of the carriers are doing their best to follow RFCs, use e-mail filtering, and perform deep packet inspection to keep themselves off of the lists. AND there seems to be some technologies that were meant to work, and cause their own sets of problems (example: allowing the end user to choose what is considered spam and blacklisting based on that). As was said before, it's not the WHY but rather how can we fix it if it's broke. The large debate seems to revolve around responsibility, or lack thereof. In our case, we are the small operator who sits in the sidelines hoping that someone larger than us, or more influential has an opinion. We participate in lists, hoping to make a difference and contribute, knowing that in a lot of cases, our opinion is just that: an opinion. I suppose that could spark a debate about joining organizations (who shall go nameless here), power to the people, etc. It seems as though a potential solution *may* revolve around ARIN/IANA having the ability to communicate an authoritative list of reassigned IP blocks back to the carriers. This could serve as a signal to remove a block from the RBL, but I'm sure there will be downfalls with doing this as well. In my specific case, I am left with a legacy block that I have to accept is going to be problematic. Simply contacting RBL operators is just not doing the trick. Most of the e-mails include links or at least an error code, but some carriers just seem to be blocking without an error, or even worse, an ACL... We will continue to remove these blocks as necessary, reassign IPs from other blocks where absolutely necessary, and ultimately hope the problem resolves itself over time. Thanks again for the very thoughtful and insightful comments, they are greatly appreciated. Regards, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com - Original Message - From: Tom Pipes tom.pi...@t6mail.com To: nanog@nanog.org Sent: Tuesday, September 8, 2009 9:57:58 AM GMT -06:00 US/Canada Central Subject: Repeated Blacklisting / IP reputation Greetings, We obtained a direct assigned IP block 69.197.64.0/18 from ARIN in 2008. This block has been cursed (for lack of a better word) since we obtained it. It seems like every customer we have added has had repeated issues with being blacklisted by DUL and the cable carriers. (AOL, ATT, Charter, etc). I understand there is a process to getting removed, but it seems as if these IPs had been used and abused by the previous owner. We have done our best to ensure these blocks conform to RFC standards, including the proper use of reverse DNS pointers. I can resolve the issue very easily by moving these customers over to our other direct assigned 66.254.192.0/19 block. In the last year I have done this numerous times and have had no further issues with them. My question: Is there some way to clear the reputation of these blocks up, or start over to prevent the amount of time we are spending with each customer troubleshooting unnecessary RBL and reputation blacklisting? I have used every opportunity to use the automated removal links from the SMTP rejections, and worked with the RBL operators directly. Most of what I get are cynical responses and promises that it will be fixed. If there is any question, we perform inbound and outbound scanning of all e-mail, even though we know that this appears to be something more relating to the block itself. Does anyone have any suggestions as to how we can clear this issue up? Comments on or off list welcome. Thanks, --- Tom Pipes T6 Broadband/ Essex Telcom Inc tom.pi...@t6mail.com
