RE: Re[2]: [Declude.JunkMail] How to get negative rounded SA result usiing SPAMC32
Thanks Sandy. Believe me, I don't mind the wait. You definition of wait is not bad! This isn't an emergency anyway, just something I'm hoping to utilize to fine-tune things and make HAM results useful. Keep up the good work! Geoff __ Geoff Varney Network Support Specialist Educational Service District 112 Ridgefield School District 360-619-1405 [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sanford Whiteman Sent: Monday, January 09, 2006 8:58 PM To: Geoff Varney Subject: Re[2]: [Declude.JunkMail] How to get negative rounded SA result usiing SPAMC32 OK, I'm on an even older version of Declude (still 1.81!) since I haven't taken the time to switch over and make any changes that are needed. Now it's time to do that! I'll test this on another server and see if anything is different then see what happens with my mail server. If I can't get any better results with this, I'll see if Declude can help. It seems from what Nick is saying that Declude is not processing negative result codes for him, either, and he's in the 2.x range. Unlikely that this was fixed in 3.x. In defense of Declude, negative result codes/errorlevels for console EXEs would typically be used for system-level errors, such as an inability to access SPAMC32.EXE due to NTFS ACLs, etc. It would not be _completely_ unreasonable, though still unfortunate, if Declude purposely does not consider negatives as application-level results. The good news is that I can work around this; the bad news is that you'll have to wait for me to update SPAMC32 toward the end of the week. By using SPAMC32's existing local threshold functionality (-lt and -ht), you can set one SPAMC32 test definition to return a result code of 1 if -100 = %SPAMD_WEIGHT% = -1. That should solve your problem. The reason you have to wait is that due to the way that SPAMC32 parses the command line (i.e. using hyphens as argument delimiters), putting a negative weight after -lt or -ht will currently be ignored. Gotta fix that. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.imprimia.com/products/software/freeutils/SPAMC32/download/release / Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.imprimia.com/products/software/freeutils/exchange2aliases/downloa d/release/ http://www.imprimia.com/products/software/freeutils/ldap2aliases/download/re lease/ --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
[Declude.JunkMail] WMF and MIME blocking
FYI from the Full Disclosure mailing list- re: WMF Preliminary testing reveals that emails containing WMF files can be blocked by filtering for the MIME-encoded WMF header. This approach works even if the file is called WORD.DOC. The string to check for is: 183GmgAA These 8 bytes appear as the first 8 bytes of a MIME-encoded WMF. Thus, blocking all emails with those bytes in will block all emails containing WMFs. This technique can be used with common spam filters. Regarding web-based WMFs, of the three browsers on this system, only IE knows what to do with WMFs. --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] WMF and MIME blocking
That would be this posting: http://lists.grok.org.uk/pipermail/full-disclosure/2006-January/041032.h tml I'm willing to bet that this information is not to be trusted, Dave. I'm confident enough and lazy enough that I'm not going to test it. Preliminary testing reveals that emails containing WMF files can be blocked by filtering for the MIME-encoded WMF header. A) If a blackhat is going to take the effort, even with the Metasploit framework, to create a malformed WMF with a trojan inside, that same blackhat will find it trivial to craft a non-compliant MIME entry in the email. Virus and spam authors ignore MIME standards anyway as a matter of course. Regarding web-based WMFs, of the three browsers on this system, only IE knows what to do with WMFs. B) This guy is presenting a very weak follow-up on ground already trod by giants. IE as a default browser will open the attachment automagically and the exploit can take place invisibly. The other browsers (Opera, Firefox, et al) will prompt the user as to whether the default application should be used to open the object. The user is then free to self-inflict the malware on themselves by clicking OK. And most users would. Andrew 8( --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[2]: [Declude.JunkMail] OT: Issues with Windows 2003 FTP service
The exact source of the problem was a limit in threads in MS SMTP, which is something like 20 per processor (not sure if hyperthreading counts in this case). . . This isn't an accurate claim. It sounds like you just haven't tried to tune MS SMTP appropriately for load/resources/growth. That's fine, since MS doesn't push you too hard on this, but please do some more lab work before deciding that MS SMTP is broken. Look into the following values: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters\PoolThreadLimit HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\MaxPercentPoolThreads HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\AdditionalPoolThreadsPerProc You also need to realize the differences between I/O completion port architecture, callbacks, and pure multithreading. It's possible to bottleneck anything, but that's an implementation problem with a given event sink, not a problem with the overall framework. . . . this limitation in the MS SMTP architecture makes it inappropriate for any full scale spam blocking solution such as Declude unless you have that application ride behind MS SMTP instead of being plugged into it. . . . Sorry, man, but that's FUD. I appreciate what you found with FTP in your config, but let's not jump over the available data. One other note. I had previously used the Windows registry hack to enable a native tarpitting feature in MS SMTP with even worse affects. The built-in tarpitting in MS SMTP will delay all 5xx error codes by the time that you set, and this included the 552 error used when messages are over the maximum size. The result of this terrible oversight is that a fair number of servers will not recognize the delayed 552 error and will requeue the oversized messages over and over again, and that eats up a lot of bandwidth real, real quick. Matt, you _know_ this isn't cause and effect, since you and I both looked at a specific server _without_ tarpitting enabled that exhibited the problem with oversize messages. There may be an additional wrinkle added by ttarpitting, but it's not the root cause of the problem. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.imprimia.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.imprimia.com/products/software/freeutils/exchange2aliases/download/release/ http://www.imprimia.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
[Declude.JunkMail] ANN: SPAMC32 (SpamAssassin SPAMC for Declude) 0.5.58 released
-- SPAMC32 Release 0.5.58 1/10/2006 * Release notes for this version: [ + Added feature] [ * Improved/changed feature ] [ - Bug fix ] [ ^ Cosmetic/naming change ] [+] Added support for negative values to '-lt' and '-ht' switches, which control client-side spam thresholds. This was necessary because Declude does not appear to properly detect negative return codes, although SPAMC32 can return them. A workaround is to use negative client thresholds, such as SPAMASSASSIN external nonzero c:\imail\declude\spamc32.exe -lt (20) -ht 0 -f choose a weight 0 This test definition will set SPAMC32 to return code 1 to Declude if the SPAMD result is between -20 and 0. NOTE: because of the way SPAMC32 parses the command line, it is necessary to express negative values in the 'spreadsheet' style that encloses them in parentheses and does not use the minus sign, i.e. (20) instead of -20 The negative value will NOT work as expected if you use the minus sign. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.imprimia.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.imprimia.com/products/software/freeutils/exchange2aliases/download/release/ http://www.imprimia.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Issues with Windows 2003 FTP service
Sandy, you have such a delicate way with words... I am only passing on what I was told by Peter from VamSoft. There is a whole thread on this where Peter confirmed the affect on FTP (but not IIS) in VamSoft's o.e.support newsgroup, which I see you found after sending this message. I personally would be surprised to see a fixed (non-tweakable) thread limit, but I can't make a judgment without some form of verification of your suggestion considering the guidance that I was given (I'm not the programmer nor the expert). This server averages only about 2% CPU utilization, so it would be a shame to have it top out so prematurely. I did previously search for "CDO_OnArrival thread", but came up with nothing at all that was useful. Looks like I should have searched for "SMTPSVC threads" instead since I did come across an article mentioning the registry parameters that you mentioned. I'll wait for Peter to reply to your suggestion. Regarding the MS SMTP tarpitting, you have to trust me on that, or at least test it yourself before suggesting that I am wrong here. This was a different issue than that other server had. The issue with that server was that MS SMTP returns 552 errors in the middle of the DATA, and while many servers support that, IMail doesn't. I have since convinced people at Ipswitch that this needs to be changed even if it isn't technically RFC compliant due to real-world conditions, and the fact that Microsoft's implementation makes more sense and should be easy enough to support. This hasn't apparently been worked into the software yet because they were in the final stages of testing IMail 2006, and this isn't a quick fix. I expect to see it happen soon though, hopefully right after they get the major issues worked out with the 2006 release. The MS SMTP tarpitting issue is one that I came across under other circumstances. When I had this turned on (not the ORF version, but the registry hack), I experienced the condition repeatedly from all sorts of servers where I was the recipient. After extensive research and testing, I found that it was this registry hack that was causing the majority of the servers to never recognize the 552 error code, and since all this did was delay the response, one has to assume that the delay of the 552 error was responsible. With this turned off (the default), I am still experiencing some issues with requeued incoming oversized E-mail, but they are much lower in number and related to the sending of 552 errors in the middle of the DATA command. Microsoft shouldn't be haphazardly delaying all 5xx errors, but instead should just be delaying the ones related to invalid recipients. This was a bad oversight on their part, and I am 100% positive about the cause. It may be almost unnoticeable to those that are running this in front of less E-mail addresses and don't actively monitor their bandwidth like I do. Due to my volume, this issue became so large that it was eating up many times my normal incoming bandwidth on a regular basis. Note that I tried to work around both MS SMTP 552 issues on incoming E-mail by setting MS SMTP to accept messages of unlimited size and then configuring IMail to signal the error, but MS SMTP hears the 552 error that IMail generates and then tries to bounce the entire oversized message through the configured Smart Host, which was the same IMail server that wouldn't accept the message in the first place. Why you would want to send a 50 MB bounce message is beyond me. I could kludge something together to bypass the problem, that's not the issue though. My point is that I have been through this stuff with a fine-toothed comb. Matt Sanford Whiteman wrote: The exact source of the problem was a limit in threads in MS SMTP, which is something like 20 per processor (not sure if hyperthreading counts in this case). . . This isn't an accurate claim. It sounds like you just haven't tried to tune MS SMTP appropriately for load/resources/growth. That's fine, since MS doesn't push you too hard on this, but please do some more lab work before deciding that MS SMTP is broken. Look into the following values: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters\PoolThreadLimit HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\MaxPercentPoolThreads HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\AdditionalPoolThreadsPerProc You also need to realize the differences between I/O completion port architecture, callbacks, and pure multithreading. It's possible to bottleneck anything, but that's an implementation problem with a given event sink, not a problem with the overall framework. . . . this limitation in the MS SMTP architecture makes it inappropriate for any full scale spam blocking solution such as Declude unless you have that application ride behind MS SMTP instead of being plugged into it. . . . Sorry, man, but
RE: [Declude.JunkMail] OT: Issues with Windows 2003 FTP service
This may be of some use...These are two of the things checked by theMicrosoft Exchange Best Practices Tool (sorry, there is no KB listed): 1) How to restrict the size of the bounce messages you generate (just like the big boys do with their postfix and sendmail MTAs, but it's possible that Exchange, not IIS uses this, and that Exchange just uses this registry key as a good place to put the value): MaxDSNSize not set Send Feedback The Microsoft Exchange Server Best Practices Analyzer Tool reads the following registry entry to determine whether the maximum size for delivery status notification (DSN) messages has been set: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSvc\Queuing\MaxDSNSize The Exchange Server Best Practices Analyzer also queries the Active Directory directory service to determine the value of the serialNumber attribute for all objects with an object class of msExchExchangeServer. If the string value includes "Version 5.5," the computer is running Microsoft Exchange Server 5.5. If the string value includes "Version 6.0," the computer is running Microsoft Exchange 2000 Server. If the string value includes "Version 6.5," the computer is running Microsoft Exchange Server 2003. If the Exchange Server Best Practices Analyzer finds that the MaxDSNSize registry value does not exist on a computer that is running Exchange 2000 Server or Exchange Server 2003, a warning is displayed. The MaxDSNSize registry value is an optional value in Exchange Server that limits the size of DSNs. This option configures Exchange to remove attachments if a message cannot be delivered. If you add this registry value, it is recommended that a value of 1024 bytes (10 megabytes) be used. Note: The MaxDSNSize registry value was introduced in Service Pack 2 (SP2) for Exchange 2000 Server, and therefore requires Exchange 2000 Server SP2 or later versions to work. Additionally, in Service Pack 1 for Exchange Server 2003 and later versions, a value of 10 megabytes is used in the absence of the MaxDSNSize registry value. If you want, you can add the MaxDSNSize registry value to override this default behavior. If you enable this option, you can save server and network resources. However, there are drawbacks to this implementation of Simple Mail Transfer Protocol (SMTP) attachment stripping. If you enable this option to strip the attachments from the non-delivery report (NDR), the details that are necessary to display the notification in the preview pane are also stripped, and the originator of the message cannot use the Send Again option. If the originator of the message tries to use the Send Again option from the NDR, the originator of the message receives the following error message: Unable to resend the message. The nondelivery report does not contain sufficient information about the original message. To resend the message, open it in your Sent Items folder, click the Actions menu, and click "Resend this message". However, the originator of the message cannot resend the message, even by using the method in the error message. Important: This article contains information about editing the registry. Before you edit the registry, make sure you understand how to restore the registry if a problem occurs. For information about how to restore the registry, view the "Restore the Registry" Help topic in Regedit.exe or Regedt32.exe. To correct this warningOpen a registry editor, such as Regedit.exe or Regedt32.exe. Navigate to: HKLM\System\CurrentControlSet\Services\SMTPSvc Right-click SMTPSvc and select New | Key. Name the new key Queuing and leave the class blank. Right-click Queuing and select New | DWORD Value. Name the new value MaxDSNSize. In the right pane, double-click the MaxDSNSize registry value and enter the size limit (in bytes) that you want in the Value data field. Messages that are larger than this value that generate an NDR do not return attachments or full message properties. Close the registry editor, and then restart the Simple Mail Transfer Protocol (SMTP) service for the change to take effect. For more information about the MaxDSNSize registry setting, see the following Microsoft Knowledge Base articles: 308303, "XCON: Option to Strip Attachments for Messages That Generate an NDR," (http://go.microsoft.com/fwlink/?linkid=3052kbid=308303) 323484, "XCON: Description of the Multipart/Report Internet Message Format in Exchange 2000," (http://go.microsoft.com/fwlink/?linkid=3052kbid=323484) 2) One possible registry key that would scale up the number of threads used by IIS (hey, it seems like a good entry): The Microsoft Exchange Server Best Practices Analyzer Tool reads the following registry entries to determine whether the maximum number of Internet Information Services (IIS) pool threads has been modified from the default value: HKEY_LOCAL_MACHINE\ System\CurrentControlSet\Services\Inetinfo\Parameters\PoolThreadLimit
Re[2]: [Declude.JunkMail] OT: Issues with Windows 2003 FTP service
I am only passing on what I was told by Peter from VamSoft. There is a whole thread on this where Peter confirmed the affect on FTP (but not IIS) in VamSoft's o.e.support newsgroup, which I see you found after sending this message. Peter is very smart developer, but not a systems guy. You probably noticed our back-and-forth about ORF's 64-bit support. During our off-list continuation, I was struck equally by his lack of facility with certain common OS features (such as COM+) and yet the expertise with which he built ORF for scaleability and performance. I don't expect him to do in-depth tweaking outside of his app (and I'm sure he didn't touch these parameters here). I personally would be surprised to see a fixed (non-tweakable) thread limit, but I can't make a judgment without some form of verification of your suggestion considering the guidance that I was given (I'm not the programmer nor the expert). Tweak the settings and open up a lot of sessions to your server; you will see that the number of worker threads (as opposed to I/O ports) that can be created rises accordingly. Regarding the MS SMTP tarpitting, you have to trust me on that, or at least test it yourself before suggesting that I am wrong here. I'm sure you _are_ seeing aggravated problems with tarpitting enabled. Yet it's clear that you've observed problems with oversize messages _whether or not_ tarpitting is enabled, and you treated that fact insufficiently, it seemed to me. From what you wrote, it sounds like running the sink that I wrote for the non-tarpitting server would solve both problems. As you may have noticed, it generates a DSN that does not include the original message, just the subject line, the message size, and the size limit. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.imprimia.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.imprimia.com/products/software/freeutils/exchange2aliases/download/release/ http://www.imprimia.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
