Re: [clamav-users] Any way to force scan as mail?
On Mar 2, 2017, at 10:43 AM, G.W. Haywood wrote: I reject everything from Hotmail (and Yahoo, Comcast, RR, Gmail and a couple of dozen other fairly big 'free' email providers) because that is much more effective and much less demanding on CPU+RAM than trying to sift out the 0.5% of half-way genuine mail from those sources using a tool like ClamAV. How nice for you. Your little world must be so orderly. For many of us, 80-95% of our legit mail comes from such sites too. ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
Hi there, On Fri, 3 Mar 2017, Reindl Harald wrote: ... do yourself a favour and click on the "raw" link Click? My mail client is called 'Alpine'. It doesn't do 'click'. ... that's a pure raw-eml with nothing HTMLified But still not the original problem message... -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
Am 02.03.2017 um 19:43 schrieb G.W. Haywood: Hi there, On Thu, 2 Mar 2017, Bowie Bailey wrote: ... Hate to say it, but you downloaded the wrong files. ... At the risk of stating the obvious, I downloaded the links that the OP gave in his post. As I said, they're HTMLified garbage. As I also said, tools are available to deal easily with mail headers if they're causing problems Not detected as mail: http://pastebin.com/LCipWJaQ do yourself a favour and click on the "raw" link http://pastebin.com/raw/LCipWJaQ that's a pure raw-eml with nothing HTMLified ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
Hi there, On Thu, 2 Mar 2017, Bowie Bailey wrote: ... Hate to say it, but you downloaded the wrong files. ... At the risk of stating the obvious, I downloaded the links that the OP gave in his post. As I said, they're HTMLified garbage. As I also said, tools are available to deal easily with mail headers if they're causing problems. But, more importantly, this is all about ClamAV failing to find ONE malicious message sent via Hotmail. The discussion would make a lot more sense if (a) malicious messages didn't outnumber the other kind by a factor of something like twenty, and (b) ClamAV (even if it could reliably find the body in a Hotmail-delivered message) were not, as a virus scanner, next to useless. Incidentally other contributors to this list have made similar points, and one said fairly recently that ClamAV couldn't find water in the ocean although I think that's rather harsh. In case you're wondering, I use ClamAV because it can use the excellent signatures maintained by Steve Basford at Sanesecurity to fight spam. Without them I would have no use for ClamAV. I reject everything from Hotmail (and Yahoo, Comcast, RR, Gmail and a couple of dozen other fairly big 'free' email providers) because that is much more effective and much less demanding on CPU+RAM than trying to sift out the 0.5% of half-way genuine mail from those sources using a tool like ClamAV. Botnets are a much bigger source; my milters spot them within a few milliseconds of their establishing a connection, and tarpit them immediately - for anything up to a few years, it depends - without any help from ClamAV, which is incapable of detecting them. As a commercial provider the OP might not feel that he can afford the luxury of rejecting mail from all the freebie providers, and he quite likely has to rely on Spamhaus et al to drop botnet connections which is a bit hit and miss at the best of times, but in my view the world (or at least his customers) would be safer if he put his effort into something that's capable of yielding better results. Either way, despite the initial reaction, I don't want to see somebody go up a blind alley if a little thought will give far more reward with much less perspiration, wafer fabrication and electricity consumption. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
> El 28/02/2017 a las 19:15, Noel Jones escribió:
>> On 2/28/2017 11:35 AM, Carlos Velasco wrote:
>>
>> Anyway, the main question remains unanswered... is there any way to force
>> the scan as mail (overriding the magic for the first recursion)?
>>
>
>
> Clam uses the daily.ftm file to decide what type of scanning to use.
> Generally, clam looks for a Received: line or a few other common
> mail headers in the first few bytes of the file. Apparently those
> common headers are too far into your file.
>
> You can create a local.ftm with your unusual headers in it to cause
> these files to be detected as an email. I don't see my notes for
> the .ftm file syntax at the moment, but I'm sure you can find
> something on google.
>
> Alternately, you can get the sanesecurity.ftm file from
> sanesecurity.com, which includes a wide variety of mail formats and
> will likely recognize your file. You don't need to use any the
> sanesecurity add-on signatures for this, but I recommend them.
Thank you very much for your reply, Noel.
You are right, in the daily.ftm are magics for Mail Files and as far as I
understand them there are some than only match from 0 to offset 1024.
1:0,1024:0a(46|66)726f6d3a20{-1024}0a(4d|6d)(49|69)(4d|6d)(45|65)2d(56|76)657273696f6e3a20:Mail
file:CL_TYPE_ANY:CL_TYPE_MAIL
1:0,1024:0a(46|66)726f6d3a20{-2048}0a(43|63)6f6e74656e742d(54|74)7970653a20:Mail
file:CL_TYPE_ANY:CL_TYPE_MAIL
1:0,1024:0a(4d|6d)(49|69)(4d|6d)(45|65)2d(56|76)657273696f6e3a20{-2048}0a(43|63)6f6e74656e742d(54|74)7970653a20:Mail
file:CL_TYPE_ANY:CL_TYPE_MAIL
1:0,1024:0a(4d|6d)6573736167652d(49|69)643a20{-1024}0a(43|63)6f6e74656e742d(54|74)7970653a20:Mail
file:CL_TYPE_ANY:CL_TYPE_MAIL
I have created a local.ftm with this line and at last file was recognized as
mail:
1:0,8192:0a(4d|6d)(49|69)(4d|6d)(45|65)2d(56|76)657273696f6e3a20{-2048}0a(43|63)6f6e74656e742d(54|74)7970653a20:Mail
file:CL_TYPE_ANY:CL_TYPE_MAIL
It's very frustrating because I know this file is an mail and I can not tell
ClamAV to not use magic and treat this file as an mail (forced).
Sadly this email file is not unusual at all, this issue is caused by a simple
email from hotmail received at a MX. :(
DKIM and a lot more of headers are surprisingly usual nowadays.
Regards,
Carlos Velasco
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
On 3/1/2017 1:00 PM, G.W. Haywood wrote: Hello again, On Wed, 1 Mar 2017, Carlos Velasco wrote: G.W. Haywood wrote: > Your conjecture is incorrect. Neither of those things is a properly > formed mail message. I'd describe them as jumbled up collections of > bits and pieces of things which might possibly once have been parts of > mail messages. Sorry but you are wrong, they are indeed real mails and properly formatted. Directly received from hotmail. I just have changed (hidden) the domains, addresses and IP addresses at the moment of publishing them. It is the magic of ClamAV (0.99.2) that does not detects mail for the first case, but it detects mails for the second case (with just 1 long header line deleted). Tested ClamAV devel version makes partial detection of mail (through MHTML). Magic of "file" works for both, detecting both as mail text: # file LCipWJaQ.txt LCipWJaQ.txt: ASCII mail text, with very long lines, with CRLF line terminators # file ZvmST7Xh.txt ZvmST7Xh.txt: ASCII mail text, with very long lines, with CRLF line terminators I've been doing this for a couple of decades, so do I know what a properly formed mail message looks like. :) The text files on which you ran 'file' and the HTML-ified garbage to which you linked in your original post are not the same things at all: laptop3:~$ >>> wget -q http://pastebin.com/ZvmST7Xh laptop3:~$ >>> file ZvmST7Xh ZvmST7Xh: HTML document, ASCII text, with very long lines, with CRLF, LF line terminators laptop3:~$ >>> wget -q http://pastebin.com/LCipWJaQ laptop3:~$ >>> file LCipWJaQ LCipWJaQ: HTML document, ASCII text, with very long lines, with CRLF, LF line terminators You owe it to anyone who might take the trouble to help you at least to provide *exactly* the data with which you are having problems - not some vague, Webserver-generated representation of it - and perhaps also to consider their replies more carefully. Hate to say it, but you downloaded the wrong files. You need to get the 'raw' version. Otherwise, you just get pastebin's website view. $ wget http://pastebin.com/raw/ZvmST7Xh $ file ZvmST7Xh ZvmST7Xh: ASCII mail text, with very long lines, with CRLF line terminators $ wget -q http://pastebin.com/raw/LCipWJaQ $ file LCipWJaQ LCipWJaQ: ASCII mail text, with very long lines, with CRLF line terminators -- Bowie ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
Hello again, On Wed, 1 Mar 2017, Carlos Velasco wrote: G.W. Haywood wrote: > Your conjecture is incorrect. Neither of those things is a properly > formed mail message. I'd describe them as jumbled up collections of > bits and pieces of things which might possibly once have been parts of > mail messages. Sorry but you are wrong, they are indeed real mails and properly formatted. Directly received from hotmail. I just have changed (hidden) the domains, addresses and IP addresses at the moment of publishing them. It is the magic of ClamAV (0.99.2) that does not detects mail for the first case, but it detects mails for the second case (with just 1 long header line deleted). Tested ClamAV devel version makes partial detection of mail (through MHTML). Magic of "file" works for both, detecting both as mail text: # file LCipWJaQ.txt LCipWJaQ.txt: ASCII mail text, with very long lines, with CRLF line terminators # file ZvmST7Xh.txt ZvmST7Xh.txt: ASCII mail text, with very long lines, with CRLF line terminators I've been doing this for a couple of decades, so do I know what a properly formed mail message looks like. :) The text files on which you ran 'file' and the HTML-ified garbage to which you linked in your original post are not the same things at all: laptop3:~$ >>> wget -q http://pastebin.com/ZvmST7Xh laptop3:~$ >>> file ZvmST7Xh ZvmST7Xh: HTML document, ASCII text, with very long lines, with CRLF, LF line terminators laptop3:~$ >>> wget -q http://pastebin.com/LCipWJaQ laptop3:~$ >>> file LCipWJaQ LCipWJaQ: HTML document, ASCII text, with very long lines, with CRLF, LF line terminators You owe it to anyone who might take the trouble to help you at least to provide *exactly* the data with which you are having problems - not some vague, Webserver-generated representation of it - and perhaps also to consider their replies more carefully. Anyway, the main question remains unanswered... is there any way to force the scan as mail (overriding the magic for the first recursion)? My original reply stands. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
On 2/28/2017 11:35 AM, Carlos Velasco wrote: > > Anyway, the main question remains unanswered... is there any way to force the > scan as mail (overriding the magic for the first recursion)? > Clam uses the daily.ftm file to decide what type of scanning to use. Generally, clam looks for a Received: line or a few other common mail headers in the first few bytes of the file. Apparently those common headers are too far into your file. You can create a local.ftm with your unusual headers in it to cause these files to be detected as an email. I don't see my notes for the .ftm file syntax at the moment, but I'm sure you can find something on google. Alternately, you can get the sanesecurity.ftm file from sanesecurity.com, which includes a wide variety of mail formats and will likely recognize your file. You don't need to use any the sanesecurity add-on signatures for this, but I recommend them. -- Noel Jones ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
>> Some days ago I stepped into a problem where ClamAV was not >> detecting a virus attached in an email. I narrowed the problem to >> Clam not detecting the file passed as a mail. I think this is >> because mail file has too many headers. > > Your conjecture is incorrect. Neither of those things is a properly > formed mail message. I'd describe them as jumbled up collections of > bits and pieces of things which might possibly once have been parts of > mail messages. Sorry but you are wrong, they are indeed real mails and properly formatted. Directly received from hotmail. I just have changed (hidden) the domains, addresses and IP addresses at the moment of publishing them. It is the magic of ClamAV (0.99.2) that does not detects mail for the first case, but it detects mails for the second case (with just 1 long header line deleted). Tested ClamAV devel version makes partial detection of mail (through MHTML). Magic of "file" works for both, detecting both as mail text: # file LCipWJaQ.txt LCipWJaQ.txt: ASCII mail text, with very long lines, with CRLF line terminators # file ZvmST7Xh.txt ZvmST7Xh.txt: ASCII mail text, with very long lines, with CRLF line terminators Anyway, the main question remains unanswered... is there any way to force the scan as mail (overriding the magic for the first recursion)? Regards, Carlos Velasco ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Any way to force scan as mail?
Hi there, On Tue, 28 Feb 2017, Carlos Velasco wrote: Is there any way to force clamscan to treat the file passed as a mail? Yes, for example you could turn it into a mail message. There are numerous tools which can do that, I would suggest something like 'formail'. Some days ago I stepped into a problem where ClamAV was not detecting a virus attached in an email. I narrowed the problem to Clam not detecting the file passed as a mail. I think this is because mail file has too many headers. Your conjecture is incorrect. Neither of those things is a properly formed mail message. I'd describe them as jumbled up collections of bits and pieces of things which might possibly once have been parts of mail messages. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
