Hi,
Well, the necessary logic seems absolutely simple:
A) parse token, up to EITHER CR or LF
B) if "CR" found, see if followed by LF, skip over CF/LF and start new line
C) if only "CR" found, and not followed by LF, skip over CR and start new
line. Set SMTP Header violation flag.
D) if "LF" found, skip over LF and start new line.
Set SMTP Header violation flag.
The key is to detect an "apparent" end of headers - no matter if it's a
proper CRLFCRLF sequence, or if it's only a CRCR or LFLF or CRLFLF
combination so that spam can be recognized as such, virus can't be hidden in
none-standard line feeds and Declude doesn't attach headers behind the body
of the message.
I'm certain that we are looking at one cause why obvious spam is currently
slipping past Declude in the current releases.
Best Regards
Andy Schmidt
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt
Sent: Friday, October 20, 2006 05:35 PM
To: declude.junkmail@declude.com
Subject: Re: SPAM-WARN: Re: [Possible Spam][Declude.JunkMail] On RFC
Violation - Declude allows attachments and Virus to pass through untouched
and unscanned
I think that the real issue is that if Declude just simply understood CR
patterns, the messages could be properly parsed, and headers could be
properly inserted.
There is a similar and longstanding issue with Declude failing to decode
base64 encoding in very long lines (i.e. +1000 or so characters long). This
also leads to failing to scan the attachments. One virus has been spreading
for the last year and a half with this symptom, and the only reason why my
Declude virus stops it is because it always sends as an EXE, but this causes
backscatter. If this was changed to do something like a ZIP file, which
most of us don't block, it could result in viruses passing straight through
Declude.
While Declude does have a new option to treat CR patterns as a
vulnerability, this isn't something that I can use since there is
unfortunately a steady, yet small, stream of legitimate messages that have
this flaw. The flaw itself doesn't represent a vulnerability, in fact this
wouldn't be a vulnerability at all if Declude understood how to parse the
messages. I can't see how fixing this flaw, or the base64 long line flaw
would take them any more than a couple of hours. I have a plug-in that I
wrote for Declude that is fully capable of understanding CR patterns as well
as long base64 code without issue.
Matt
Michael Thomas - Mathbox wrote:
David,
In my opinion, which others may not share, Declude should detect all
RFC/MIME violations and flag them in some manner. There exist quite
a few
that are common to spam messages, but not flagged by Declude.
However, that
is a totally different subject than the point of my test. The RFC
violation
was simply a symptom. Some admins might not choose to delete or
block
messages with that construction. There exist well-known web sites
that
generate response email messages where one or two lines out 50-100
are
missing one of the Cr/Lf pair characters.
The point is that for many (maybe most) people on this list, Declude
is the
lock and keys for securing email passing through their mail servers.
Declude
trusts that the email message will be well-formed. Because of that
mis-placed trust, Declude did not reliably detect any attachment,
regardless
of type, and therefore did not invoke the scanners. Note that this
test was
performed on Declude version 3.1.1. Maybe, the new gateway product
is not
quite so trusting. I have no idea. We are an ISP and would not pay
the fees
associated with ISP use of the gateway product.
In any event, this is a well-known issue and has been for some time.
I
reported this issue to Declude and the list some time ago regarding
spam not
being scanned because of this issue. At that time, I was so focused
on the
spam problem, I did not think about the attachment/virus side of the
issue,
which should have been obvious to me.
Michael Thomas
Mathbox
978-683-6718
1-877-MATHBOX (Toll Free)
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of David Sullivan
Sent: Friday, October 20, 2006 3:51 PM
To: declude.junkmail@declude.com
Subject: SPAM-WARN: Re: [Possible Spam][Declude.JunkMail] On
RFC Violation - Declude allows attachments and Virus to pass
through untouched and unscanned
Hello Michael,
Thanks for the great research.
Wouldn't this be the purpose of
Vulnerability detection in Declude? "Declude detects
mal-formed
messages that can allow viruses to be hidden from email
server virus
scanners."
We treat all vulnerabilities as viruses, send the notice and
86 the
message.
-David
Thursday, October 19, 2006, 10:52:25 PM, you wrote:
MTM> Hi All,
MTM> Well, when responding on declude.junkmail@declude.com
to
Will about RFC
MTM> violations, I said I would test this and I did.
MTM> --------------------
MTM> While writing this message, I happened to think about
attachments. It would
MTM> appear to me, that there is an implied possibility for
attachments and
MTM> therefore viruses to pass through undetected. All that
should be required is
MTM> that the lines that make up the entire email, including
the attachment
MTM> section, be terminated with line feeds instead of
carriage return/line feed
MTM> pairs. Under such condition, Declude would see only one
line and not find
MTM> the relevant sections. I will test this possibility.
MTM> --------------------
MTM> Tested: Declude v3.1.1 for IMail
MTM> As it happens, my suspicions were accurate. I wrote a
script that could be
MTM> modified to remove either the carriage-returns or the
line-feeds from a
MTM> message file. I then created a message in Outlook
Express, added an
MTM> executable file (uptime.exe) as an attachment and saved
it in my Draft
MTM> folder. I then dragged that message to the same
location
as the script and
MTM> renamed it to match the file name in the script
(Rfc.eml) I ran the script,
MTM> which stripped the carriage-returns and produced
Rfc2.eml. I renamed
MTM> Rfc2.eml to RfcNoCr.eml. In the script, I then changed
vbCr to vbLf and ran
MTM> it again, which stripped the line-feeds and produced
Rfc2.eml. I renamed
MTM> Rfc2.eml to RfcNoLf.eml.
MTM> Now, to get IIS SMTP to actually process the file, you
must edit each file
MTM> and remove the single Cr or Lf and press the Enter Key,
producing a CrLf
MTM> pair after the To field and the From field. I also
added
the string "No Cr"
MTM> to the end of the subject of RfcNoCr.eml and added No
Lf
to the subject of
MTM> RfcNoLf.eml. So for example change:
MTM> --------------------
MTM> From: "Michael Thomas - Mathbox" <[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED]> [Cr]To:
MTM> "[EMAIL PROTECTED]" <mailto:[EMAIL PROTECTED]>
[Cr]Subject: Test Attachment
Pass-Through on RFC
MTM> Violation[Cr]<line continues>
MTM> --------------------
MTM> Change To
MTM> --------------------
MTM> From: "Michael Thomas - Mathbox" <[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED]>
MTM> To: "[EMAIL PROTECTED]" <mailto:[EMAIL PROTECTED]>
MTM> Subject: Test Attachment Pass-Through on RFC Violation
No Cr[Cr]<line
continues>>
MTM> --------------------
MTM> Now it so happens, a long time ago, I wrote a couple of
tests to detect
MTM> these RFC violations, so first I had to disable them in
my GLOBAL.CFG, which
MTM> I did by commenting them out. Note that I also BAN the
.EXE extension and I
MTM> left that enabled.
MTM> Now copy and paste the two files into the pickup
directory of your favorite
MTM> IIS SMTP pickup directory. Viola, you just passed an
executable through
MTM> Declude and through your mail server. That executable
could very well have
MTM> been a virus.
MTM> Note that Declude detected RfcNoLf.eml as [Outlook 'CR'
Vulnerability]. Ok
MTM> good.
MTM> But Declude let RfcNoCr.eml pass straight through
without calling the virus
MTM> scanners, because Declude did NOT see an attachment.
Also, because Declude
MTM> did not see an attachment, Declude did not ban the .EXE
extension.
MTM> Here are the log entries from RfcNoLf.eml
MTM> --------------------
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Scanning
Time: 218ms
MTM> [kernel=31 user=187]
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Virus
scanner 1 reports exit
MTM> code of 0
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Virus
detected. Not continuing
MTM> with remaining scanners.
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd 0:
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Starting
EXT check .
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd
MTM> C:\IMAIL\spool\proc\work\D1b2101b7000083ba.vir\*.*
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd 0
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Deleted
MTM> C:\IMAIL\spool\proc\work\D1b2101b7000083ba.vir\0.
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd
report.txt
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Deleted
MTM>
C:\IMAIL\spool\proc\work\D1b2101b7000083ba.vir\report.txt.
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd
han=13e9c0 b=False
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd File(s)
are INFECTED [[Outlook
MTM> 'CR' Vulnerability]: 0]
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd High
code=23.
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd AV
returned 23
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Scanned:
CONTAINS A VIRUS
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd From:
[EMAIL PROTECTED] To:
MTM> [EMAIL PROTECTED] [incoming from XX.XXX.XXX.X]
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Subject:
Test Attachment
MTM> Pass-Through on RFC Violation No Lf
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Skipping
non-AV E-mail
MTM> BANnotify.eml
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd
MTM> C:\IMAIL\Declude\postmaster.eml
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd Starting
E-mail file
MTM> C:\IMAIL\Declude\postmaster.eml
MTM> 10/19/2006 20:41:23.471 q1b2101b7000083ba.smd
C:\IMAIL\IMail1.exe -h
MTM> "mathbox.com" -t "[EMAIL PROTECTED]"
<mailto:[EMAIL PROTECTED]> -u
"[EMAIL PROTECTED]" <mailto:[EMAIL PROTECTED]> -s
MTM> "Mathbox Email Virus Scanning detected and quarantined
a
virus" -f
MTM> "C:\IMAIL\spool\proc\work\D1b2101b7000083ba.sm0"
MTM> 10/19/2006 20:41:23.487 q1b2101b7000083ba.smd TempName
=
MTM> C:\IMAIL\Declude\postmaster.eml
MTM> --------------------
MTM> Here are the log entries from RfcNoCr.eml
MTM> --------------------
MTM> 10/19/2006 20:41:10.690 q1b2101da000083bb.smd Setting
Scan File 1 to
MTM> C:\Progra~1\FSI\F-Prot\FPcmd.exe /TYPE /SILENT /SERVER
/NOMEM /ARCHIVE
MTM> /NOBOOT /DUMB /REPORT=report.txt.
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd CFG:
Setting report parse 1 to
MTM> Infection.
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Setting
Scan File 2 to
MTM> C:\imail\declude\runclamscan.exe log=3
MTM> C:\clamav-devel\bin\clamscan.exe
MTM> --quiet --no-summary --tempdir=c:\tmp\
MTM> --database=C:\clamav-devel\share\clamav\ --max-ratio=0
--mbox -l report.txt.
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd CFG:
Setting report parse 2 to
MTM> FOUND.
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Setting
virus directory to:
MTM> C:\IMAIL\spool\virus
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Incoming
E-mail scanning
MTM> turned ON
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Outgoing
E-mail scanning
MTM> turned ON
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Setting
AVAFTERJM to ON.
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Setting
MAXATONCE to 20.
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Setting
scanner timeout to 120
MTM> seconds
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Setting
AUTOFORGE to OFF.
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Scanner 0
Virus Codes: 3 6 8 9
MTM> 10 . OK Codes:
MTM> 10/19/2006 20:41:10.721 q1b2101da000083bb.smd Scanner 1
Virus Codes: 1 . OK
MTM> Codes:
MTM> 10/19/2006 20:41:10.908 q1b2101da000083bb.smd Skip
Extensions: GIF TXT MPG
MTM> PNG
MTM> 10/19/2006 20:41:10.955 q1b2101da000083bb.smd 48 Ban
Extensions: ADE ADP ASD
MTM> ASP BAS BAT BIN CAB CHM CMD COM CPL CRT DLL EXE HLP HTA
HTO INF INS ISP JS
MTM> JSC JSE KSH LNK MDB MDE MSI OCX PCD PIF REG SCF SCR SCT
SHB SHS SYS VB VBE
MTM> VBS VBX VSMACROS VXD WSC WSF WSH
MTM> 10/19/2006 20:41:11.002 q1b2101da000083bb.smd Virus Pro
Registered
MTM> 10/19/2006 20:41:11.018 q1b2101da000083bb.smd Starting
locality check
MTM> (sender=mathbox.com; nr=1 ca=off). nHas=1.
MTM> 10/19/2006 20:41:11.018 q1b2101da000083bb.smd
[EMAIL PROTECTED] [0-0] is
MTM> local domain1 viaFM
MTM> 10/19/2006 20:41:11.018 q1b2101da000083bb.smd Ending
locality check
MTM> (cached), sender=local.
MTM> 10/19/2006 20:41:11.018 q1b2101da000083bb.smd Local
host
= mathbox.com
MTM> 10/19/2006 20:41:11.018 q1b2101da000083bb.smd
MTM> [EMAIL PROTECTED] Offset=5
MTM> Flags=1
MTM> 10/19/2006 20:41:11.033 q1b2101da000083bb.smd Msgid:
MTM> 10/19/2006 20:41:11.049 q1b2101da000083bb.smd Subject:
Test Attachment
MTM> Pass-Through on RFC Violation No Cr
MTM> --------------------
MTM> Here is the script to strip Cr or Lf, just change the
vbCr below to vbLf.
MTM> Just save it as:
MTM> Rfc.vbs
MTM> --------------------
MTM> Dim InFile
MTM> Dim OutFile
MTM> Dim Fso, File
MTM> Dim AllText
MTM> InFile = "Rfc.eml"
MTM> OutFile = "Rfc2.eml"
MTM> Set Fso = CreateObject("Scripting.FileSystemObject")
MTM> If Fso.FileExists( InFile ) = True Then
MTM> Set File = Fso.OpenTextFile( InFile, 1, False, 0
)
MTM> AllText = File.ReadAll
MTM> File.Close
MTM> Set File = Nothing
MTM> AllText = Replace( AllText, vbCr, "" )
MTM> Set File = Fso.OpenTextFile( OutFile, 2, True, 0
)
MTM> File.Write AllText
MTM> File.Close
MTM> Set File = Nothing
MTM> End If
MTM> Set Fso = Nothing
MTM> --------------------
MTM> Finally, if you want to test for these RFC violations,
see
MTM> http://www.mathbox.com/NoCrTest/NoCrTest.zip
MTM> Michael Thomas
MTM> Mathbox
MTM> 978-683-6718
MTM> 1-877-MATHBOX (Toll Free)
MTM> ---
MTM> This E-mail came from the Declude.JunkMail mailing
list. To
MTM> unsubscribe, just send an E-mail to
[EMAIL PROTECTED], and
MTM> type "unsubscribe Declude.JunkMail". The archives can
be found
MTM> at http://www.mail-archive.com.
--
Best regards,
David mailto:[EMAIL PROTECTED]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED],
and
type "unsubscribe Declude.JunkMail". The archives can be
found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe
Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
