> On Oct 14, 2019, at 4:39 PM, Nick via mailop <mailop@mailop.org> wrote: > > On 2019-10-14 15:47 BST, Steve Atkins via mailop wrote: >> On 14/10/2019 14:58, Nick via mailop wrote: >>> >>> My question remains unanswered. Why not treat each ip address on >>> its own merits? Is it technically infeasible, too expensive, less >>> convenient, or what? >> >> There's more than one reason. Some are technical. >> >> But also expending the effort to track an ever-changing >> set of reputation-IP pairs against hostile actors > > Isn't that effort already being expended every day? Do you mean it > would increase by some too-costly multiple? > >> is subsidizing providers whose business model relies on allowing >> customers to send malicious traffic, including unwanted email, to >> the customers of those who are being asked to expend that effort. > > That's the first mention I've seen of a subsidy - could you elaborate > on how that works?
Providers, whether they be ESPs, hosting companies, VPS outfits really want to make money. The model is simplest to explain for an ESP, but all the same mechanics apply to many other hosting arrangements. The ESP can make money by providing an excellent service, at a steep price, to companies with high budgets and expectations to match. Princesses. They can make money by selling to naive/greedy businesses who kinda just want to batch and blast a list they've acquired over the years or bought somewhere. Hobbits & Kobolds. And they can make money by selling service to really pretty bad spammers, whether they be the criminal end who negotiate "you won't cut us off, and will cover for us" or the giant corporation or political party whose address acquisition practices are terrible but who just bring in too much money to terminate. Lizardmen. If you sell to Princesses you're spending a lot on customer acquisition and professional services to bring them onboard, and there aren't *that* many of them out there. Losing a Princess will have a bad effect on your annual bonus. Lizardmen are easy to find and onboard - it's keeping them off your network that's tricky - and they're a sure source of revenue, at least as long as they're getting some mail delivered via you and they don't have to spend too much time defending themselves. They have a lot of money and no shame. Hobbits and Kobolds of various shapes and sizes make up the majority of the available customers. They're more price sensitive and have more reasonable expectations than the Princesses. A solid source of revenue, as long as you have a bunch of them and you're able to service them without too much expenditure on your part. If you're sending mail for more than a few lizardmen then your delivery will increasingly take a hit, both mechanically and because ISPs are going to be less and less likely to take your word for "It's OK, the lizardmen accidentally mailed a suppression list, we'll help them improve their practices." As that happens you'll find you can't acquire new Princesses and existing Princesses will need more (expensive) handholding and will begin to migrate their business elsewhere. Your middle ground of customers will start drifting to be more Kobold and less Hobbit. Your level of service to them will likely degrade, but the Kobolds care less about that than the Hobbits. And the Kobolds who hear and see that you tolerate Lizardmen on your network will preferentially come to you, as you gather a reputation of being an ESP that won't be "mean" to them As you lose your Princesses you're increasingly relying on the Lizardmen for the backbone of your revenue, and the Kobolds seem to keep breeding. You still have quite a few existing Hobbits and even a Princess or two, but not enough to make this quarters projections. Bring on a few more Lizardmen, make it to the end of Q2, but your reputation is getting pretty bad and mail's not really getting delivered well. Maybe you can tech and social engineer your way through that situation, for years, or maybe your Lizardmen start leaving for the next vulnerable ESP. If you want to keep your high-value Princesses, and bring more of them to your network you need to avoid that situation. Keep the worst of the Kobolds and most of the Lizardmen off your network altogether, and work fairly hard to mitigate the damage caused by the ones that are left. BUT. What if you could have the terrible behaviour of the Lizardmen not impact your Princesses? You could take the /22 of IP addresses you've been sending all your customers traffic through, intermingled, and split it up. You could put the Lizardmen in their own /24, the Princesses in their own /24 and divide your Kobolds and Hobbits over another couple of /24s. Then tell ISPs "Yeah, there's some bad Lizardmen, but they're all in this /24 over _here_, so maybe you could just block that, and let all our other mail through?" (This was, in every detail, the business model of one ESP, Topica, in the early 2000s. Possibly even including literal Lizardmen, I'm not sure.) This doesn't work quite as well as the ESP wants it to, as the rest of the Internet says a) "you're taking money from the people who are spewing spam, why should we be nice to the rest of your customers?" and b) "but sure, we'll block all the Lizardmen - and maybe the Kobolds and Hobbits too". The next approach is "Let individual customers live by the IP reputation and die by the IP reputation!". Put each customer on their own IP address and hope that only individual IP addresses get blocked - which will encourage the Hobbits, at least, to clean up their practices so as to avoid getting blocked. Back when blocks were manually applied - and there was a significant amount of manual effort in adding or removing them - this kinda works. The Lizardmen IP addresses get blocked. As long as the ESP does some minimal amount of work on policing their customers and social engineering blacklist operators most ISPs will just block the egregious stuff they notice, rather than escalating to broader blocks. But the Internet does still say a) "We block Lizardmen when we notice them, but new ones keep popping up on your network - or are they the same Lizardmen, just wearing fake mustaches on new IPs?" and b) "Hey, why are *we* subsidizing *your* business model of let-lizardmen-spam-but-dont-block-princesses? We do all the work of identifying the bad customers as they pop up, and blocking them, and you just keep taking the Lizardmen's money without having to spend any money or risk any Princess accounts!" [*] A very, very small subset of the Internet is actually making those decisions, though. You could mail them a box of steaks, and maybe they'd special case your traffic. But then comes automation of blocking. Thresholds. Statistics. Eventually machine learning. When blocking is based on "mail that looks like *this* and fails to hit *these* metrics gets blocked, mail that exceeds *these* metrics" gets delivered" instead of being a manually maintained "Somebody's Spam List" it's a lot harder to relax filters for a specific friend. Mailing a box of steaks to the person running the filters won't affect their behaviour (it didn't really affect it when they were configured manually, but the ESPs didn't believe that and, hey, steaks are a cheap investment). Blowing a bunch of filthy Lizardman money on fancy booze, generic coke and cheap hookers in Vegas won't affect the filters. Even regular payoffs to ISP staff won't let them special case the filtering for *your* ESP. You can get inside information to help you get through the filters, for a while. But it's not going to give you enough of an edge to do it for long. And whining at ISP staff, whether it be on news.admin.net-abuse.email, at MAAWG or ESPC, on Zorch, on SPAM-L, on NANOG or, yes, on mailop doesn't work any more either. "Our filters are designed to keep our customers happy. We've tuned the algorithms, min-maxed the options, balanced false positives against false negatives and we make really good delivery decisions based on the thousands of metrics we track, from the perspective of optmizing our users happiness. The *vast* majority of wanted 1:1 mail gets delivered, most wanted bulk mail gets delivered, very little spam and hardly any malware." It's a system that's hard to game, because it measures things that are strongly correlated with whether a recipient wants to see the email. If you want your mail to be delivered you have to send mail people want, and not look like people who send mail recipients don't want. You can't social-engineer your way into good email delivery unless the people you're social engineering are your recipients - and then it's just good marketing. Cheers, Steve [*] There's the subsidy. _______________________________________________ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
