Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
On 2/21/2018 11:44 AM, Dianne Skoll wrote: On Wed, 21 Feb 2018 16:35:27 + Karol Augustinwrote: I think the point here might be that if Google acted promptly on abuse spammers would stop using shorteners. True, that might happen. OTOH, I see about as many spams with bit.ly shorteners as goo.gl shorteners which is not what one might expect if bit.ly were really that much more proactive than goo.gl. I'm sure mileage may vary - but I'm seeing about 10X the abuse for goo.gl right now as I see from bit.ly. Also, when I do random checks on a handful of abused bitly shortners, a high percentage of them are already terminated. But when I do random checks of abused goo.gl redirectors, most of them are still operational. (I'm referring to redirectors found in spams within the previous few days of when I checked them, with at least hours having gone by since the message was sent - I know that sounds anecdotal - but as I've been researching this in the past weeks, that pattern keeps happening). One thing that could potentially make those numbers different - is when you compare one system that blocks MUCH spam at the perimeter based on the sending IP, such as blocking all Zen-listed spams before DATA while another system might capture ALL messages and process them all. The latter is what my system does. That also might explain the difference in stats? -- Rob McEwen https://www.invaluement.com
Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
On Wed, 21 Feb 2018 16:35:27 + Karol Augustinwrote: > I think the point here might be that if Google acted promptly on abuse > spammers would stop using shorteners. True, that might happen. OTOH, I see about as many spams with bit.ly shorteners as goo.gl shorteners which is not what one might expect if bit.ly were really that much more proactive than goo.gl. Regards, Dianne.
Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
On 2018-02-21 16:31, Dianne Skoll wrote: > On Wed, 21 Feb 2018 11:29:00 -0500 > Rob McEwenwrote: > >> Nevertheless, it is a shame to have to shift more of the burden onto >> spam filters to do more work (some of which requires MORE latency) - >> in order to partly mitigate Google's failure to prevent/correct the >> abuse. > > Yes, I agree. On the other hand, IMO a spam filter should block messages > that point to a taken-down shortened URL. Although such messages may > not be downright dangerous, they are still annoying and are still spam. > There's no way of avoiding the work. > I think the point here might be that if Google acted promptly on abuse spammers would stop using shorteners. Karol -- Karol Augustin ka...@augustin.pl http://karolaugustin.pl/ +353 85 775 5312
Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
On Wed, 21 Feb 2018 11:29:00 -0500 Rob McEwenwrote: > Nevertheless, it is a shame to have to shift more of the burden onto > spam filters to do more work (some of which requires MORE latency) - > in order to partly mitigate Google's failure to prevent/correct the > abuse. Yes, I agree. On the other hand, IMO a spam filter should block messages that point to a taken-down shortened URL. Although such messages may not be downright dangerous, they are still annoying and are still spam. There's no way of avoiding the work. Regards, Dianne.
Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
On 2/21/2018 11:11 AM, Dianne Skoll wrote: I guess I misinterpreted: "...such automated lookups could also put a huge extra burden on Google's servers..." from Message-IdOh yeah, I'd forgotten about that part. it was a more minor point. But as I think back on my thought processes at the time I typed those words - I was envisioning what would happen if ALL ISPs and hosters and spam filtering vendors, SA installations etc... ALL started doing all of those lookups. But yeah, maybe that would still be a drop in the bucket compared to all that Google does. Nevertheless, it is a shame to have to shift more of the burden onto spam filters to do more work (some of which requires MORE latency) - in order to partly mitigate Google's failure to prevent/correct the abuse. In contrast, at this time, bit.ly is running circles around Google in terms of quickly shutting down their abused redirectors. I know this isn't easy, but there is definitely room for improvement. But my larger point in that overall post you quoted from, was my concern about one organization doing high volume lookups from a single server getting blocked or captcha'd. -- Rob McEwen https://www.invaluement.com +1 (478) 475-9032
Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
On Wed, 21 Feb 2018 10:58:17 -0500 Rob McEwenwrote: > On 2/21/2018 10:37 AM, Dianne Skoll wrote: > > The concern voiced in another email about overloading Google's > > infrastructure is quite charming and quaint. > My concern was NEVER about overloading google. I guess I misinterpreted: "...such automated lookups could also put a huge extra burden on Google's servers..." from Message-Id > My concern was about Google auto-blocking or throwing a captcha at > very high volume and automated lookups. That is a HUGE difference. That's a concern, but you can mitigate it slightly by caching the results of URL expansion. Also, I suspect most URL-shorteners expect to be bombarded with requests and are engineered accordingly. I have seen a CAPTCHA thrown up from the search page, but I've never seen a goo.gl HEAD request reply with anything other than 404 or 301. Regards, Dianne.
Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
On 2/21/2018 10:37 AM, Dianne Skoll wrote: The concern voiced in another email about overloading Google's infrastructure is quite charming and quaint. My concern was NEVER about overloading google. My concern was about Google auto-blocking or throwing a captcha at very high volume and automated lookups. That is a HUGE difference. -- Rob McEwen https://www.invaluement.com
Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))
Dianne Skoll skrev den 2018-02-21 16:37: We do a HEAD request and it works on most URL shorteners. The concern voiced in another email about overloading Google's infrastructure is quite charming and quaint. +1 some with icla could add this to spamasssassin with https://github.com/smfreegard/DecodeShortURLs so if yes problem is solved imho, but should we consider all _URIHOSTS_ as shortners ? unless thay are confirmed not to do this