Re: Dealing with links to malicious documents

[Markus Clardy]

I had created a plugin to read the headers (just make a HEAD request) of
all URIs in an email, and then you can make tests based off them. An
example of this would be to look for specific mimetypes at the end of the
link, so for example, see if the mimetype is Application/msword.

You can find it here: https://github.com/m50/spamassassin_uriheader

I will warn you, this may not necessarily be a great idea, as you may get
abuse notifications. I also need to make a few changes to it, but I haven't
touched it in a while.

First thing I want to do is whitelist certain domains from being header
checked.

Secondly, is replace the User Agent with one from a legitimate browser
(spoofing a browser), just in case.


But if you are interested in it, you can take a look.

On Tue, Mar 13, 2018 at 10:03 PM, John Hardin  wrote:

> On Tue, 13 Mar 2018, Bill Cole wrote:
>
> On 13 Mar 2018, at 14:21 (-0400), John Hardin wrote:
>>
>> d) Don't accept emails from outside your organization that link to hosted
>>> documents. The document needs to be attached, so that it can be scanned.
>>> Unfortunately this is not feasible if you're not a (at least
>>> semi-)monolithic organization where you can apply such policies.
>>>
>>
>> Also not feasible if any users subscribe to this list or most technical
>> discussion mailing lists. For example, here you are likely to get links
>> into the SA Wiki or to KAM's rules. On the Postfix list it is a rare week
>> that does not have multiple links to the DEBUG_README file posted.
>>
>
> I don't count a plain text file as a "document" in this context.
>
> The example provided was apparently to a directory (URL ending in '/') but
>> redirected to a .doc.
>>
>
> This of course is the weakness with that option.
>
>
> --
>  John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
>  jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
>  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
> ---
>   The problem is when people look at Yahoo, slashdot, or groklaw and
>   jump from obvious and correct observations like "Oh my God, this
>   place is teeming with utter morons" to incorrect conclusions like
>   "there's nothing of value here".-- Al Petrofsky, in Y! SCOX
>
> ---
>  Tomorrow: Albert Einstein's 139th Birthday
>



-- 
 - Markus

Re: Dealing with links to malicious documents

[John Hardin]

On Tue, 13 Mar 2018, Bill Cole wrote:


On 13 Mar 2018, at 14:21 (-0400), John Hardin wrote:

d) Don't accept emails from outside your organization that link to hosted 
documents. The document needs to be attached, so that it can be scanned. 
Unfortunately this is not feasible if you're not a (at least 
semi-)monolithic organization where you can apply such policies.


Also not feasible if any users subscribe to this list or most technical 
discussion mailing lists. For example, here you are likely to get links into 
the SA Wiki or to KAM's rules. On the Postfix list it is a rare week that 
does not have multiple links to the DEBUG_README file posted.


I don't count a plain text file as a "document" in this context.

The example provided was apparently to a directory (URL ending in '/') 
but redirected to a .doc.


This of course is the weakness with that option.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The problem is when people look at Yahoo, slashdot, or groklaw and
  jump from obvious and correct observations like "Oh my God, this
  place is teeming with utter morons" to incorrect conclusions like
  "there's nothing of value here".-- Al Petrofsky, in Y! SCOX
---
 Tomorrow: Albert Einstein's 139th Birthday

Re: Dealing with links to malicious documents

[Bill Cole]

On 13 Mar 2018, at 14:21 (-0400), John Hardin wrote:

d) Don't accept emails from outside your organization that link to 
hosted documents. The document needs to be attached, so that it can be 
scanned. Unfortunately this is not feasible if you're not a (at least 
semi-)monolithic organization where you can apply such policies.


Also not feasible if any users subscribe to this list or most technical 
discussion mailing lists. For example, here you are likely to get links 
into the SA Wiki or to KAM's rules. On the Postfix list it is a rare 
week that does not have multiple links to the DEBUG_README file posted. 
The example provided was apparently to a directory (URL ending in '/') 
but redirected to a .doc.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Currently Seeking Steady Work: https://linkedin.com/in/billcole

Re: Dealing with links to malicious documents

[Rupert Gallagher]

Hardin has it right, especially for invoices.

"The document needs to be attached, so that it can be scanned."

Re: Dealing with links to malicious documents

[Pedro David Marco]

 

   
>a) Link following
>Whether it is only for url shorteners or for all links, simulating a click 
>could give us info on what will happen, but has implications when the website 
>interprets that like a click from the user and updates their database in >some 
>way such as unsubscribing a user.
Be carefull with one time links  Download only headers may be a good idea

Another idea may be to donwload headers with different user-agents . If we get 
different responses.

PedroD

 
  

Re: Dealing with links to malicious documents

[Leandro]

2018-03-13 15:13 GMT-03:00 Olivier Coutu :

> In the last few months, we have seen an increase of generic emails (e.g.
> regarding unpaid invoices) being sent with links to infected legitimate
> websites hosting malware. This malware often comes in the form of docs with
> macros e.g. https://pastebin.com/VHz41RUL
>
> In a lot of cases, neither the sender nor the URL are listed in any
> blacklists at send time, and we are looking into ways to deal with these
> links. We have developed some heuristics based on the text but this is more
> reactive than proactive and the spams often are very similar to legitimate
> emails. Ideally we would be able to see what is *really* behind these
> links.
>
> The technologies we know exist are:
>
> a) Link following
> Whether it is only for url shorteners or for all links, simulating a click
> could give us info on what will happen, but has implications when the
> website interprets that like a click from the user and updates their
> database in some way such as unsubscribing a user.
>
>
Our customers decides if the filter must follow all redirections or follow
just shorteners redirections. If they choose all redirections, the filter
will download any file to AV scan, if exists.

We make it very clear what is the filter implications if it follow all
redirects. Most have chosen to let the filter follow all redirects, even
though they may activate unwanted functions on the links. For them, is
preferable to take the risk of activating unwanted functions than to run
the risk of receiving a ransomware.

I believe that web designers need to get used to the idea of programming
double confirmation functions because it is increasingly frequent the
filters that follow all the redirects, because of all the abuse we are
witnessing. Just put a confirm button at web function is enough to avoid
unwanted function.

Re: Dealing with links to malicious documents

[John Hardin]

On Tue, 13 Mar 2018, Alex wrote:


Hi,

On Tue, Mar 13, 2018 at 2:21 PM, John Hardin  wrote:

On Tue, 13 Mar 2018, Olivier Coutu wrote:


In the last few months, we have seen an increase of generic emails (e.g.
regarding unpaid invoices) being sent with links to infected legitimate
websites hosting malware. This malware often comes in the form of docs with
macros e.g. https://pastebin.com/VHz41RUL

In a lot of cases, neither the sender nor the URL are listed in any
blacklists at send time, and we are looking into ways to deal with these
links. We have developed some heuristics based on the text but this is more
reactive than proactive and the spams often are very similar to legitimate
emails. Ideally we would be able to see what is /really/ behind these links.

The technologies we know exist are:

a) Link following
Whether it is only for url shorteners or for all links, simulating a click
could give us info on what will happen, but has implications when the
website interprets that like a click from the user and updates their
database in some way such as unsubscribing a user.

b) Link rewriting
Rewrite the link so that it is analysed by the anti-spam provider at
click-time. Costly to implement and breaks message integrity/DKIM. Even
after 24h, a lot of these infected websites are not listed on blacklists.
This method also has privacy implications.

c) DNS-based approaches
Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to
block queries to malicious websites. Our tests indicate that this does not
work very well for the aforementioned infected websites. It might work well
for C servers but we feel like that is a bit late to avoid an infection.

Are there other solutions that we have not thought of? Are any of you
having trouble with these types of links?



d) Don't accept emails from outside your organization that link to hosted
documents. The document needs to be attached, so that it can be scanned.
Unfortunately this is not feasible if you're not a (at least
semi-)monolithic organization where you can apply such policies.


I don't think denying access to direct downloads in email is really a
policy that would work in any moderately sized organization. I tried
looking at this in the recent past, and it would block a ton of
legitimate email.


Really? That surprises me a bit. PDFs I could see, but document files?


Also, in this particular case, you don't know that the link downloads
something until you actually click on it or follow it.


Ah, yes, that wouldn't work.


e) if you are a (semi-)monolithic organization that uses a boundary proxy to
control web access, then add such URLs to that proxy's blocklist until the
contents can be scanned, or so that the proxy does the redirect-through-AV
automatically (not sure if that will work, though).


I think what some providers do is wrap the URL and at some point later
(after perhaps the domain or URL itself is listed by an RBL) disable
the link for when the recipient clicks the link or views the email. I
don't know how (or whether) they always break DKIM in the process.
That would be interesting to find out.



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Gun Control is marketed to the public using the appealing delusion
  that violent criminals will obey the law.
---
 Tomorrow: Albert Einstein's 139th Birthday

Re: Dealing with links to malicious documents

[Axb]

On 03/13/2018 07:13 PM, Olivier Coutu wrote:
In the last few months, we have seen an increase of generic emails (e.g. 
regarding unpaid invoices) being sent with links to infected legitimate 
websites hosting malware. This malware often comes in the form of docs 
with macros e.g. https://pastebin.com/VHz41RUL


In a lot of cases, neither the sender nor the URL are listed in any 
blacklists at send time, and we are looking into ways to deal with these 
links. We have developed some heuristics based on the text but this is 
more reactive than proactive and the spams often are very similar to 
legitimate emails. Ideally we would be able to see what is /really/ 
behind these links.


The technologies we know exist are:

a) Link following
Whether it is only for url shorteners or for all links, simulating a 
click could give us info on what will happen, but has implications when 
the website interprets that like a click from the user and updates their 
database in some way such as unsubscribing a user.


b) Link rewriting
Rewrite the link so that it is analysed by the anti-spam provider at 
click-time. Costly to implement and breaks message integrity/DKIM. Even 
after 24h, a lot of these infected websites are not listed on 
blacklists. This method also has privacy implications.


c) DNS-based approaches
Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to 
block queries to malicious websites. Our tests indicate that this does 
not work very well for the aforementioned infected websites. It might 
work well for C servers but we feel like that is a bit late to avoid 
an infection.


Are there other solutions that we have not thought of? Are any of you 
having trouble with these types of links?


Why not write a bunch of SA URI rules and/or ClamAV sigs to handle them.

Re: Dealing with links to malicious documents

[Alex]

Hi,

On Tue, Mar 13, 2018 at 2:21 PM, John Hardin  wrote:
> On Tue, 13 Mar 2018, Olivier Coutu wrote:
>
>> In the last few months, we have seen an increase of generic emails (e.g.
>> regarding unpaid invoices) being sent with links to infected legitimate
>> websites hosting malware. This malware often comes in the form of docs with
>> macros e.g. https://pastebin.com/VHz41RUL
>>
>> In a lot of cases, neither the sender nor the URL are listed in any
>> blacklists at send time, and we are looking into ways to deal with these
>> links. We have developed some heuristics based on the text but this is more
>> reactive than proactive and the spams often are very similar to legitimate
>> emails. Ideally we would be able to see what is /really/ behind these links.
>>
>> The technologies we know exist are:
>>
>> a) Link following
>> Whether it is only for url shorteners or for all links, simulating a click
>> could give us info on what will happen, but has implications when the
>> website interprets that like a click from the user and updates their
>> database in some way such as unsubscribing a user.
>>
>> b) Link rewriting
>> Rewrite the link so that it is analysed by the anti-spam provider at
>> click-time. Costly to implement and breaks message integrity/DKIM. Even
>> after 24h, a lot of these infected websites are not listed on blacklists.
>> This method also has privacy implications.
>>
>> c) DNS-based approaches
>> Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to
>> block queries to malicious websites. Our tests indicate that this does not
>> work very well for the aforementioned infected websites. It might work well
>> for C servers but we feel like that is a bit late to avoid an infection.
>>
>> Are there other solutions that we have not thought of? Are any of you
>> having trouble with these types of links?
>
>
> d) Don't accept emails from outside your organization that link to hosted
> documents. The document needs to be attached, so that it can be scanned.
> Unfortunately this is not feasible if you're not a (at least
> semi-)monolithic organization where you can apply such policies.

I don't think denying access to direct downloads in email is really a
policy that would work in any moderately sized organization. I tried
looking at this in the recent past, and it would block a ton of
legitimate email.

Also, in this particular case, you don't know that the link downloads
something until you actually click on it or follow it.

> e) if you are a (semi-)monolithic organization that uses a boundary proxy to
> control web access, then add such URLs to that proxy's blocklist until the
> contents can be scanned, or so that the proxy does the redirect-through-AV
> automatically (not sure if that will work, though).

I think what some providers do is wrap the URL and at some point later
(after perhaps the domain or URL itself is listed by an RBL) disable
the link for when the recipient clicks the link or views the email. I
don't know how (or whether) they always break DKIM in the process.
That would be interesting to find out.

Re: Dealing with links to malicious documents

[John Hardin]

On Tue, 13 Mar 2018, Olivier Coutu wrote:

In the last few months, we have seen an increase of generic emails (e.g. 
regarding unpaid invoices) being sent with links to infected legitimate 
websites hosting malware. This malware often comes in the form of docs with 
macros e.g. https://pastebin.com/VHz41RUL


In a lot of cases, neither the sender nor the URL are listed in any 
blacklists at send time, and we are looking into ways to deal with these 
links. We have developed some heuristics based on the text but this is more 
reactive than proactive and the spams often are very similar to legitimate 
emails. Ideally we would be able to see what is /really/ behind these links.


The technologies we know exist are:

a) Link following
Whether it is only for url shorteners or for all links, simulating a click 
could give us info on what will happen, but has implications when the website 
interprets that like a click from the user and updates their database in some 
way such as unsubscribing a user.


b) Link rewriting
Rewrite the link so that it is analysed by the anti-spam provider at 
click-time. Costly to implement and breaks message integrity/DKIM. Even after 
24h, a lot of these infected websites are not listed on blacklists. This 
method also has privacy implications.


c) DNS-based approaches
Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to block 
queries to malicious websites. Our tests indicate that this does not work 
very well for the aforementioned infected websites. It might work well for 
C servers but we feel like that is a bit late to avoid an infection.


Are there other solutions that we have not thought of? Are any of you having 
trouble with these types of links?


d) Don't accept emails from outside your organization that link to hosted 
documents. The document needs to be attached, so that it can be scanned. 
Unfortunately this is not feasible if you're not a (at least 
semi-)monolithic organization where you can apply such policies.


e) if you are a (semi-)monolithic organization that uses a boundary proxy 
to control web access, then add such URLs to that proxy's blocklist until 
the contents can be scanned, or so that the proxy does the 
redirect-through-AV automatically (not sure if that will work, though).



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Gun Control is marketed to the public using the appealing delusion
  that violent criminals will obey the law.
---
 Tomorrow: Albert Einstein's 139th Birthday

Dealing with links to malicious documents

[Olivier Coutu]

In the last few months, we have seen an increase of generic emails (e.g. 
regarding unpaid invoices) being sent with links to infected legitimate 
websites hosting malware. This malware often comes in the form of docs 
with macros e.g. https://pastebin.com/VHz41RUL


In a lot of cases, neither the sender nor the URL are listed in any 
blacklists at send time, and we are looking into ways to deal with these 
links. We have developed some heuristics based on the text but this is 
more reactive than proactive and the spams often are very similar to 
legitimate emails. Ideally we would be able to see what is /really/ 
behind these links.


The technologies we know exist are:

a) Link following
Whether it is only for url shorteners or for all links, simulating a 
click could give us info on what will happen, but has implications when 
the website interprets that like a click from the user and updates their 
database in some way such as unsubscribing a user.


b) Link rewriting
Rewrite the link so that it is analysed by the anti-spam provider at 
click-time. Costly to implement and breaks message integrity/DKIM. Even 
after 24h, a lot of these infected websites are not listed on 
blacklists. This method also has privacy implications.


c) DNS-based approaches
Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to 
block queries to malicious websites. Our tests indicate that this does 
not work very well for the aforementioned infected websites. It might 
work well for C servers but we feel like that is a bit late to avoid 
an infection.


Are there other solutions that we have not thought of? Are any of you 
having trouble with these types of links?


Thanks.
- Olivier