Re: post-delivery virus scan

2016-11-10 Thread Dmitry Melekhov 

10.11.2016 16:47, Frank Elsner пишет:

On Wed, 9 Nov 2016 15:36:33 -0600 Brad Koehn wrote:

   [ ... ]


To help detect and remove the infected messages after they’ve been delivered to 
users’ mailboxes, I created a small script that iterates the INBOX and Junk 
mailbox directories, scans recent messages for viruses, and deletes them if 
found. The source of my script (run via cron) is here: 
https://gitlab.koehn.com/snippets/9

Bad idea. The user may already taken the action needed for infection. And what 
about legal aspects?

Is it legal to redistribute malware in Germany? :-D

In my country (Germany), information suppression would be punishable.


I guess main problem here is that this is not on access scan, i.e. even 
if virus can be already detected by newer virus database, it can be 
accessed by user before it rescanned.

Re: post-delivery virus scan

2016-11-10 Thread Stephan Bosch 



Op 10-11-2016 om 12:25 schreef Brad Koehn:

On Nov 10, 2016, at 3:38 AM, Stephan Bosch  wrote:

Op 11/10/2016 om 10:05 AM schreef Teemu Huovila:

On 09.11.2016 23:36, Brad Koehn wrote:

I’m wondering if there’s a better way to scan recent messages and eradicate 
them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? 
Looking for suggestions.
The removal should if possible be done with the doveadm cli tool or using the 
doveadm http api.

Still, Dovecot should handle external removal of messages gracefully.
What exactly happens?

On Dovecot 2.2.6.0:
Nov 10 10:35:13 ds dovecot: imap(user): Error: Recent flags state corrupted for 
mailbox Junk
Nov 10 10:35:13 ds dovecot: imap(user): Error: 
/var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now 
inconsistent
Nov 10 10:35:13 ds dovecot: imap(user): IMAP session state is inconsistent, 
please relogin. in=6212 out=49396


OK, so at least it doesn't panic anymore in the last release. Also, the 
mailbox is fixed upon relogin. To prevent the remaining errors from 
occurring, i.e. to gracefully remove messages, you can use the doveadm 
expunge command (it has a man page).


Regards,

Stephan.

Re: post-delivery virus scan

2016-11-10 Thread Brad Koehn 
Turns out the technical part of your reasoning is correct: MUAs that have 
downloaded the message don’t get any updates, and hold onto the infected 
message. No legal ramifications here; it’s my personal server, and it’s in the 
US. Strange to think that deleting the content of a message would somehow be 
worse than deleting the content and the headers.  


> On Nov 10, 2016, at 6:47 AM, Frank Elsner  wrote:
> 
> On Wed, 9 Nov 2016 15:36:33 -0600 Brad Koehn wrote:
> 
>  [ ... ]
> 
>> To help detect and remove the infected messages after they’ve been delivered 
>> to users’ mailboxes, I created a small script that iterates the INBOX and 
>> Junk mailbox directories, scans recent messages for viruses, and deletes 
>> them if found. The source of my script (run via cron) is here: 
>> https://gitlab.koehn.com/snippets/9
> 
> Bad idea. The user may already taken the action needed for infection. And 
> what about legal aspects?
> In my country (Germany), information suppression would be punishable.
> 
> 
> Just my 0.02 €, Frank

Re: post-delivery virus scan

2016-11-10 Thread Frank Elsner 
On Wed, 9 Nov 2016 15:36:33 -0600 Brad Koehn wrote:

  [ ... ]

> To help detect and remove the infected messages after they’ve been delivered 
> to users’ mailboxes, I created a small script that iterates the INBOX and 
> Junk mailbox directories, scans recent messages for viruses, and deletes them 
> if found. The source of my script (run via cron) is here: 
> https://gitlab.koehn.com/snippets/9

Bad idea. The user may already taken the action needed for infection. And what 
about legal aspects?
In my country (Germany), information suppression would be punishable.


Just my 0.02 €, Frank

Re: post-delivery virus scan

2016-11-10 Thread Brad Koehn 
I’ve decided to try this approach. I’ve updated my script as follows:

#!/bin/bash

# Scan junk folders for messages containing  viruses we didn't have definitions 
# for when the mail was received. Truncate the body of infected messages and
# replace the body with a message.

cd /var/mail

for dir in $( find . \( -name Junk -o -name INBOX \) -type d ) ; do
  files=$( find "$dir" -type f -name u.\* -mtime -14 -print )
  for file in $files ; do
/usr/local/bin/clamdscan --quiet --fdpass "$file"
if [ $? -eq 1 ] ; then
  sed -i '/^$/,$d' "$file"
  echo "\r\n\r\n[The body of this message contained a virus and was 
deleted.]" >> "$file"
fi
  done
done

We’ll see if that does the trick.

> On Nov 9, 2016, at 6:12 PM, mick crane  wrote:
> 
> On 2016-11-09 21:36, Brad Koehn wrote:
>> I have discovered that many times the virus definitions I use for
>> scanning messages (ClamAV, with the unofficial signatures
>> http://sanesecurity.com/usage/linux-scripts/) are updated some time
>> after my server has received an infected email. It seems the virus
>> creators are trying to race the virus definition creators to see who
>> can deliver first; more than half of the infected messages are found
>> after they’ve been delivered. Great.
>> To help detect and remove the infected messages after they’ve been
>> delivered to users’ mailboxes, I created a small script that iterates
>> the INBOX and Junk mailbox directories, scans recent messages for
>> viruses, and deletes them if found. The source of my script (run via
>> cron) is here: https://gitlab.koehn.com/snippets/9
>> Unfortunately Dovecot doesn’t like it if messages are deleted (dbox)
>> out from under it. I tried a doveadm force-resync on the folder
>> containing the messages, but it seems Dovecot is still unhappy. At
>> least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would
>> panic and coredump when it discovered messages had been deleted.
>> I’m wondering if there’s a better way to scan recent messages and
>> eradicate them so the Dovecot isn’t upset when it happens. Maybe using
>> doveadm search? Looking for suggestions.
> 
> leave an empty message behind with the same name as deleted message ?
> 
> 
> 
> 
> -- 
> key ID: 0x4BFEBB31

Re: post-delivery virus scan

2016-11-10 Thread Brad Koehn 

> On Nov 10, 2016, at 3:38 AM, Stephan Bosch  wrote:
> 
> Op 11/10/2016 om 10:05 AM schreef Teemu Huovila:
>> 
>> On 09.11.2016 23:36, Brad Koehn wrote:
>>> I have discovered that many times the virus definitions I use for scanning 
>>> messages (ClamAV, with the unofficial signatures 
>>> http://sanesecurity.com/usage/linux-scripts/) are updated some time after 
>>> my server has received an infected email. It seems the virus creators are 
>>> trying to race the virus definition creators to see who can deliver first; 
>>> more than half of the infected messages are found after they’ve been 
>>> delivered. Great. 
>>> 
>>> To help detect and remove the infected messages after they’ve been 
>>> delivered to users’ mailboxes, I created a small script that iterates the 
>>> INBOX and Junk mailbox directories, scans recent messages for viruses, and 
>>> deletes them if found. The source of my script (run via cron) is here: 
>>> https://gitlab.koehn.com/snippets/9
>>> 
>>> Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out 
>>> from under it. I tried a doveadm force-resync on the folder containing the 
>>> messages, but it seems Dovecot is still unhappy. At least on the new 
>>> version (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when 
>>> it discovered messages had been deleted. 
>>> 
>>> I’m wondering if there’s a better way to scan recent messages and eradicate 
>>> them so the Dovecot isn’t upset when it happens. Maybe using doveadm 
>>> search? Looking for suggestions. 
>> The removal should if possible be done with the doveadm cli tool or using 
>> the doveadm http api.
> 
> Still, Dovecot should handle external removal of messages gracefully.
> What exactly happens?
> 
> Regards,
> 
> Stephan.

On Dovecot 2.2.5:
Nov  9 14:32:11 ds postfix/anvil[13298]: statistics: max cache size 2 at Nov  9 
14:23:08
Nov  9 14:32:29 ds dovecot: imap(user): Error: Recent flags state corrupted for 
mailbox Junk
Nov  9 14:32:29 ds dovecot: imap(user): Error: 
/var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now 
inconsistent
Nov  9 14:32:29 ds dovecot: imap(user): Panic: Message count decreased
Nov  9 14:32:29 ds dovecot: imap(user): Error: Raw backtrace: 
/usr/local/lib/dovecot/libdovecot.so.0(+0x89cc0) [0x7f0b64641cc0] -> 
/usr/local/lib/dovecot/libdovecot.so.0(+0x89d9e) [0x7f0b646
41d9e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0b645e4165] -> 
dovecot/imap() [0x42259c] -> dovecot/imap(imap_sync_more+0x104) [0x422f14] -> 
dovecot/imap() [0x410720] -> do
vecot/imap() [0x4108d1] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x52147) [0x7f0b64917147] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xe2) 
[0x7f0b64654992]
-> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x93) 
[0x7f0b64655d83] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) 
[0x7f0b64654b45] -> /usr/l
ocal/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f0b64654cf8] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f0b645ea243] -> dovecot/imap(main+0x312) [0x40c612
] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f0b64214f45] 
-> dovecot/imap() [0x40c780]
Nov  9 14:32:30 ds dovecot: imap(bkc): Fatal: master: service(imap): child 8456 
killed with signal 6 (core dumped)


On Dovecot 2.2.6.0:
Nov 10 10:35:13 ds dovecot: imap(user): Error: Recent flags state corrupted for 
mailbox Junk
Nov 10 10:35:13 ds dovecot: imap(user): Error: 
/var/mail/user_dbox/mailboxes/Junk/dbox-Mails/dovecot.index reset, view is now 
inconsistent
Nov 10 10:35:13 ds dovecot: imap(user): IMAP session state is inconsistent, 
please relogin. in=6212 out=49396

Re: post-delivery virus scan

2016-11-10 Thread Stephan Bosch 
Op 11/10/2016 om 10:05 AM schreef Teemu Huovila:
>
> On 09.11.2016 23:36, Brad Koehn wrote:
>> I have discovered that many times the virus definitions I use for scanning 
>> messages (ClamAV, with the unofficial signatures 
>> http://sanesecurity.com/usage/linux-scripts/) are updated some time after my 
>> server has received an infected email. It seems the virus creators are 
>> trying to race the virus definition creators to see who can deliver first; 
>> more than half of the infected messages are found after they’ve been 
>> delivered. Great. 
>>
>> To help detect and remove the infected messages after they’ve been delivered 
>> to users’ mailboxes, I created a small script that iterates the INBOX and 
>> Junk mailbox directories, scans recent messages for viruses, and deletes 
>> them if found. The source of my script (run via cron) is here: 
>> https://gitlab.koehn.com/snippets/9
>>
>> Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out 
>> from under it. I tried a doveadm force-resync on the folder containing the 
>> messages, but it seems Dovecot is still unhappy. At least on the new version 
>> (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it 
>> discovered messages had been deleted. 
>>
>> I’m wondering if there’s a better way to scan recent messages and eradicate 
>> them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? 
>> Looking for suggestions. 
> The removal should if possible be done with the doveadm cli tool or using the 
> doveadm http api.

Still, Dovecot should handle external removal of messages gracefully.
What exactly happens?

Regards,

Stephan.

Re: post-delivery virus scan

2016-11-10 Thread Teemu Huovila 


On 09.11.2016 23:36, Brad Koehn wrote:
> I have discovered that many times the virus definitions I use for scanning 
> messages (ClamAV, with the unofficial signatures 
> http://sanesecurity.com/usage/linux-scripts/) are updated some time after my 
> server has received an infected email. It seems the virus creators are trying 
> to race the virus definition creators to see who can deliver first; more than 
> half of the infected messages are found after they’ve been delivered. Great. 
> 
> To help detect and remove the infected messages after they’ve been delivered 
> to users’ mailboxes, I created a small script that iterates the INBOX and 
> Junk mailbox directories, scans recent messages for viruses, and deletes them 
> if found. The source of my script (run via cron) is here: 
> https://gitlab.koehn.com/snippets/9
> 
> Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from 
> under it. I tried a doveadm force-resync on the folder containing the 
> messages, but it seems Dovecot is still unhappy. At least on the new version 
> (2.2.26.0) it doesn’t crash; 2.2.25 would panic and coredump when it 
> discovered messages had been deleted. 
> 
> I’m wondering if there’s a better way to scan recent messages and eradicate 
> them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? 
> Looking for suggestions. 
The removal should if possible be done with the doveadm cli tool or using the 
doveadm http api.

br,
Teemu Huovila
> 
> 
> 
> 
> ---
> Brad 
>

Re: post-delivery virus scan

2016-11-09 Thread mick crane 

On 2016-11-09 21:36, Brad Koehn wrote:

I have discovered that many times the virus definitions I use for
scanning messages (ClamAV, with the unofficial signatures
http://sanesecurity.com/usage/linux-scripts/) are updated some time
after my server has received an infected email. It seems the virus
creators are trying to race the virus definition creators to see who
can deliver first; more than half of the infected messages are found
after they’ve been delivered. Great.

To help detect and remove the infected messages after they’ve been
delivered to users’ mailboxes, I created a small script that iterates
the INBOX and Junk mailbox directories, scans recent messages for
viruses, and deletes them if found. The source of my script (run via
cron) is here: https://gitlab.koehn.com/snippets/9

Unfortunately Dovecot doesn’t like it if messages are deleted (dbox)
out from under it. I tried a doveadm force-resync on the folder
containing the messages, but it seems Dovecot is still unhappy. At
least on the new version (2.2.26.0) it doesn’t crash; 2.2.25 would
panic and coredump when it discovered messages had been deleted.

I’m wondering if there’s a better way to scan recent messages and
eradicate them so the Dovecot isn’t upset when it happens. Maybe using
doveadm search? Looking for suggestions.



leave an empty message behind with the same name as deleted message ?




--
key ID: 0x4BFEBB31

post-delivery virus scan

2016-11-09 Thread Brad Koehn 
I have discovered that many times the virus definitions I use for scanning 
messages (ClamAV, with the unofficial signatures 
http://sanesecurity.com/usage/linux-scripts/) are updated some time after my 
server has received an infected email. It seems the virus creators are trying 
to race the virus definition creators to see who can deliver first; more than 
half of the infected messages are found after they’ve been delivered. Great. 

To help detect and remove the infected messages after they’ve been delivered to 
users’ mailboxes, I created a small script that iterates the INBOX and Junk 
mailbox directories, scans recent messages for viruses, and deletes them if 
found. The source of my script (run via cron) is here: 
https://gitlab.koehn.com/snippets/9

Unfortunately Dovecot doesn’t like it if messages are deleted (dbox) out from 
under it. I tried a doveadm force-resync on the folder containing the messages, 
but it seems Dovecot is still unhappy. At least on the new version (2.2.26.0) 
it doesn’t crash; 2.2.25 would panic and coredump when it discovered messages 
had been deleted. 

I’m wondering if there’s a better way to scan recent messages and eradicate 
them so the Dovecot isn’t upset when it happens. Maybe using doveadm search? 
Looking for suggestions. 




---
Brad