Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Rob McEwen]

On 3/10/2018 11:43 AM, Matus UHLAR - fantomas wrote:

On 3/10/2018 11:22 AM, Matus UHLAR - fantomas wrote:
this is apparently not the case of one url redirector (shortener) 
points to

another shortener.

I really hope that the DecodeShortURLs only checks fopr redirection 
at those

known redirectors (shorteners), not each http->https shortener and only
evaluates redirection between them, ignoring http->https redirects


On 10.03.18 11:32, Rob McEwen wrote:
But also keep in mind that it is NOT rare for the initial shortner 
found in a spam... to redirect to a spammer's page (that isn't a URL 
shortner), then THAT page then redirects to the spammer's OTHER page 
(that isn't a URL shortner)... where the FIRST one is NOT blacklisted 
very many places... but the second page (that is often the final 
destination)... *is* heavily blacklisted. Often, the first page is 
hosted at a legit site that was hacked into by criminal spammers - 
and is HARDER for blacklists to list due to their good reputation. 


isn't this thread and the plugin discussed about shorteners?
I am aware of other possibilities to do redirects and about 
consequencies of

following them.
note that for following the redirect, the destination must be 
configured via

url_shortener directive.
what I wonder is, if there's real value in following the redirects.
(and if the following stops when the URL is blacklisted, since we're done
here).



The URL shortner follower plugin that Steve Freegard wrote... ALREADY 
follows my suggestions about following non-shortner redirects (after the 
initial shortner redirect). (except I'm not sure it already does my 
https suggestion?) ...and for good reason. You're tremendously 
undervaluing or just not understanding my last post. I suggestion you 
re-read it a couple of times. This tactic I mentioned that spammers 
use... is COMMON, and not following my last suggestion opens up a big 
loophole for spammers. Such spammers would be delighted by your last 
comment... and Steve Freegard went a different route for good reason. 
Keep in mind, I haven't veered off-topic because we're STILL talking 
about a possible chain of redirects that was still STARTED by a URL 
shortner.


--
Rob McEwen
https://www.invaluement.com
+1 (478) 475-9032

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Matus UHLAR - fantomas]

On 3/10/2018 11:22 AM, Matus UHLAR - fantomas wrote:
this is apparently not the case of one url redirector (shortener) 
points to

another shortener.

I really hope that the DecodeShortURLs only checks fopr redirection 
at those

known redirectors (shorteners), not each http->https shortener and only
evaluates redirection between them, ignoring http->https redirects


On 10.03.18 11:32, Rob McEwen wrote:
But also keep in mind that it is NOT rare for the initial shortner 
found in a spam... to redirect to a spammer's page (that isn't a URL 
shortner), then THAT page then redirects to the spammer's OTHER page 
(that isn't a URL shortner)... where the FIRST one is NOT blacklisted 
very many places... but the second page (that is often the final 
destination)... *is* heavily blacklisted. Often, the first page is 
hosted at a legit site that was hacked into by criminal spammers - 
and is HARDER for blacklists to list due to their good reputation. 


isn't this thread and the plugin discussed about shorteners?
I am aware of other possibilities to do redirects and about consequencies of
following them.
note that for following the redirect, the destination must be configured via
url_shortener directive. 


what I wonder is, if there's real value in following the redirects.
(and if the following stops when the URL is blacklisted, since we're done
here).

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Posli tento mail 100 svojim znamim - nech vidia aky si idiot
Send this email to 100 your friends - let them see what an idiot you are

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Rob McEwen]

On 3/10/2018 11:22 AM, Matus UHLAR - fantomas wrote:
this is apparently not the case of one url redirector (shortener) 
points to

another shortener.

I really hope that the DecodeShortURLs only checks fopr redirection at 
those

known redirectors (shorteners), not each http->https shortener and only
evaluates redirection between them, ignoring http->https redirects 



But also keep in mind that it is NOT rare for the initial shortner found 
in a spam... to redirect to a spammer's page (that isn't a URL 
shortner), then THAT page then redirects to the spammer's OTHER page 
(that isn't a URL shortner)... where the FIRST one is NOT blacklisted 
very many places... but the second page (that is often the final 
destination)... *is* heavily blacklisted. Often, the first page is 
hosted at a legit site that was hacked into by criminal spammers - and 
is HARDER for blacklists to list due to their good reputation. Then the 
2nd final destination page is just a heavily blacklisted spammer's 
throwaway domain. Therefore, there is some value to following the 
redirects a little further than what you suggest, and then collecting 
all of those host names or domains, checking ALL of them against 
URI/domain blacklists. (within reason... after too many redirects, it is 
better to just stop and add points to the spam score)


--
Rob McEwen
https://www.invaluement.com
+1 (478) 475-9032

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Matus UHLAR - fantomas]

On 3/10/2018 3:20 AM, Matus UHLAR - fantomas wrote:

do you have an example of any chained redirection not suspicious?


On 10.03.18 11:04, Rob McEwen wrote:
I haven't examined the code for that plugin very much (yet!) but 
one type of very common redirect that is very innocent... is the fact 
that a MASSIVE percentage of web sites have switched to all SSL sites 
in the past few years, due to Google now punishing http and/or 
rewarding https... in the search engine rankings. But there are still 
many links and redirectors pointing to the non-SSL versions of those 
sites, which is then typically redirected to the SSL version. 


this is apparently not the case of one url redirector (shortener) points to
another shortener.

I really hope that the DecodeShortURLs only checks fopr redirection at those
known redirectors (shorteners), not each http->https shortener and only
evaluates redirection between them, ignoring http->https redirects

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I don't have lysdexia. The Dog wouldn't allow that.

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Rob McEwen]

On 3/10/2018 3:20 AM, Matus UHLAR - fantomas wrote:
do you have an example of any chained redirection not suspicious? 



I haven't examined the code for that plugin very much (yet!) but one 
type of very common redirect that is very innocent... is the fact that a 
MASSIVE percentage of web sites have switched to all SSL sites in the 
past few years, due to Google now punishing http and/or rewarding 
https... in the search engine rankings. But there are still many links 
and redirectors pointing to the non-SSL versions of those sites, which 
is then typically redirected to the SSL version. Therefore, if the code 
for this plugin (and others using this tactic) doesn't do this 
already... it should probably not count THAT particular redirect as a 
spam indicator, when counting the total number of redirects.


--
Rob McEwen
https://www.invaluement.com

Re: razor?

[RW]

On Sat, 10 Mar 2018 09:39:20 +0100
Matus UHLAR - fantomas wrote:


> >>>For example those scores were for a totally legit email that had
> >>>some screenshots embedded in the email...  
> 
> some screenshots? afaik razor only work on text parts, so short mail
> is quite possible to be detected (as some people report image-only
> spam)

As I said, razor uses a combination of URI domains and text size.

Very short emails are all counted as the same size, which makes them
more likely to FP, but an image-only spam, without a URI, cannot be
listed in razor.

Re: Spammers, IPv6 addresses, and dnsbls

[Leandro]

>
> On 02.03.18 10:12, Leandro wrote:
>
>> If the spammer uses the same domain at rDNS, when rotating IPs, our system
>> will list each new IP at first DNSBL query.
>>
>
> do you verify synthetic rDNS too? when do you blacklist whole /64 ?
>
> I mean: there's 2^64 (18446744073709551616) IPv6 addresses in /64 range and
> there's 2^64 (18446744073709551616) of /64 IPv6 ranges in the IPv6
> namespace.
>


No. Our system lists a /64 for some cases, but not check each rDNS.
It just needs a sample large enough to list the entire /64.



>
> blacklisting either is not in possibilities of present systems.
> I'm curious whether you have any plan what to do when someone starts
> abusing
> IPv6 space and how does the plan look like.
>

Our system uses a CIDR oriented database. This means that system can be
clustering all IPv6 addresses for a minimal equivalent CIDR list.

Example:

show() => []
add("2001:db8::0") => ADDED
show() => ["2001:db8::/128"]
add("2001:db8::1") => JOINED
show() => ["2001:db8::/127"]
add("2001:db8::2") => ADDED
show() => ["2001:db8::/127", "2001:db8::2/128"]
add("2001:db8::3") => JOINED
show() => ["2001:db8::/126"]
listed("2001:db8::1") => TRUE
listed("2001:db8::4") => FALSE
overlap("2001:db8::0/64") => OVERLAPPED
show() => ["2001:db8::/64"]
remove("2001:db8::7fff") => EXTRACTED
show() => ["2001:db8::/114", "2001:db8::4000/115",
"2001:db8::6000/116", "2001:db8::7000/117",
"2001:db8::7800/118", "2001:db8::7c00/119",
"2001:db8::7e00/120", "2001:db8::7f00/121",
"2001:db8::7f80/122", "2001:db8::7fc0/123",
"2001:db8::7fe0/124", "2001:db8::7ff0/125",
"2001:db8::7ff8/126", "2001:db8::7ffc/127",
"2001:db8::7ffe/128", "2001:db8::8000/113",
"2001:db8::1000:0/100", "2001:db8::100:0/104",
"2001:db8::10:0/108", "2001:db8::1:0/112",
"2001:db8::2000:0/99", "2001:db8::200:0/103",
"2001:db8::20:0/107", "2001:db8::2:0/111",
"2001:db8::4000:0/98", "2001:db8::400:0/102",
"2001:db8::40:0/106", "2001:db8::4:0/110",
"2001:db8::8000:0/97", "2001:db8::800:0/101",
"2001:db8::80:0/105", "2001:db8::8:0/109",
"2001:db8::1000:0:0/84", "2001:db8::100:0:0/88",
"2001:db8::10:0:0/92", "2001:db8::1:0:0/96",
"2001:db8::2000:0:0/83", "2001:db8::200:0:0/87",
"2001:db8::20:0:0/91", "2001:db8::2:0:0/95",
"2001:db8::4000:0:0/82", "2001:db8:0:0:0:400:0:0/86",
"2001:db8::40:0:0/90", "2001:db8::4:0:0/94",
"2001:db8::8000:0:0/81", "2001:db8::800:0:0/85",
"2001:db8::80:0:0/89", "2001:db8::8:0:0/93",
"2001:db8::1000:0:0:0/68", "2001:db8::100:0:0:0/72",
"2001:db8::10:0:0:0/76", "2001:db8::1:0:0:0/80",
"2001:db8::2000:0:0:0/67", "2001:db8::200:0:0:0/71",
"2001:db8::20:0:0:0/75", "2001:db8::2:0:0:0/79",
"2001:db8::4000:0:0:0/66", "2001:db8::400:0:0:0/70",
"2001:db8::40:0:0:0/74", "2001:db8::4:0:0:0/78",
"2001:db8::8000:0:0:0/65", "2001:db8::800:0:0:0/69",
"2001:db8::80:0:0:0/73", "2001:db8::8:0:0:0/77"]
add("2001:db8::7fff") => JOINED
show() => ["2001:db8::/64"]

This solution helps to minimize this type of abuse.


> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> There's a long-standing bug relating to the x86 architecture that
> allows you to install Windows.   -- Matthew D. Fuller
>

Re: razor?

[Matus UHLAR - fantomas]

On Fri, 9 Mar 2018 11:09:40 -0300
Robert Boyl wrote:

Just wondering, whats your thoughts on Razor?


razor is great at spam detection.


It says on their site " Detection is done with statistical and
randomized signatures that efficiently spot mutating spam content. "

For example those scores were for a totally legit email that had some
screenshots embedded in the email...


some screenshots? afaik razor only work on text parts, so short mail is
quite possible to be detected (as some people report image-only spam)


Also, how to report FP?


razor-revoke -d -dl=2 -f false-positives

where "false-positives" is a mbox file format.

On 09.03.18 09:26, David Jones wrote:
RAZOR like DCC and PYZOR shouldn't be used as a sole source of 
determining spam. 


especially DCC, since it measures bulkiness, not spamminess.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Matus UHLAR - fantomas]

On 07.03.18 10:59, sha...@shanew.net wrote:

Just FYI, it does add 3.0 points as soon as it sees any chaining at
all.  The other 5.0 points get added at 10 redirections.  That said,
I think you're guess is right that redirections start to look really
suspicious after just 3 or 4.


do you have an example of any chained redirection not suspicious?


On Sat, 3 Mar 2018, @lbutlr wrote:


On Feb 26, 2018, at 09:55, sha...@shanew.net wrote:


This is why the DecodeShortURLs plugin has an explicit limit of 10
lookups (and penalizes such with a total of 8 points).


I’d guess more than one redirect is highly suspicious and more than two is 
probably a waste of time, just score 5.0 and be done with it.

Has anyone done any analysis on multi-redirects?


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
How does cat play with mouse? cat /dev/mouse

Re: Spammers, IPv6 addresses, and dnsbls

[Matus UHLAR - fantomas]

On 02.03.18 09:58, Leandro wrote:


Hi Danilele! Our DNSBL works with individual /128 IPv6 addresses:

http://spfbl.net/en/dnsbl/

Even if the provider is offering less then /64 to customers, our DNSBL can
list IPv6 of each one.



2018-03-02 10:08 GMT-03:00 Matus UHLAR - fantomas :

how/who do you list when spammer starts rotating IPs in assigned /64
range?


On 02.03.18 10:12, Leandro wrote:

If the spammer uses the same domain at rDNS, when rotating IPs, our system
will list each new IP at first DNSBL query.


do you verify synthetic rDNS too? when do you blacklist whole /64 ?

I mean: there's 2^64 (18446744073709551616) IPv6 addresses in /64 range and
there's 2^64 (18446744073709551616) of /64 IPv6 ranges in the IPv6
namespace.

blacklisting either is not in possibilities of present systems.
I'm curious whether you have any plan what to do when someone starts abusing
IPv6 space and how does the plan look like.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
There's a long-standing bug relating to the x86 architecture that
allows you to install Windows.   -- Matthew D. Fuller