[sniffer] Re: Beta
Pete, I am attempting to get caught on the latest beta and just have a few questions. I noticed Sniffer is now called a different way in the Declude config files, is that correct? On the last release (running persistent), we have numerous entries in the declude.cfg file labeled: SNIFFER-TRAVEL external047 C:\IMail\Declude\Sniffer\WeightGate.exe -12 %WEIGHT% 19 C:\IMail\Declude\Sniffer\snifferlic.exe codehere 20 However, it appears the categories are going away (posted in some previous messages) and there is a since of urgency needed in upgrading as these won't be populated any longer soon. I take it we run the persistent mode the same way, but have a different hook into Declude? Thanks for the aid and understanding. Keith -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, October 15, 2007 10:05 PM To: Message Sniffer Community Subject: [sniffer] Re: Beta Hello Phillip, Monday, October 15, 2007, 6:39:22 PM, you wrote: I just tried installing the beta and it appeared to work too well as it was sending all the mail to the spam box. I am not sure if this was due to the bad rule that was just replaced or something I was doing wrong. The bad rule turned out not to be as bad as we suspected -- so I don't think that was the problem. Also the bad rule has been gone for a while so you're not likely to have it in your rulebase. I am currently running in the persistent mode. I set the xml file to point to the correct paths in the 4 places and started up the sniffer server. I then changed the bat file that the agent calls to run snifclient.exe file licenseID %1 is this the correct format? I am still using an old vopmail 5 mail server. The correct way to call SNFClient.exe is either just like you call the older SNF (compatibility mode): SNFClient.exe authenticationxx scanthis.msg I'm guessing from your notes and memory SNFClient.exe authenticationxx %1 --- The other way you can call SNFClient.exe to perform a scan is simply: SNFClient.exe scantthis.msg --- If you run SNFClient.exe without any parameters it will remind you how it can be used. If you called it with: SNFClient.exe file licenseid %1 Then that would be too many parameters so you would probably get an error and possibly a nonzero result. I may simply be misinterpreting your notes - but if you did something like this in your script (batch file) then it's possible the result would be to hold every message. At the moment I switched back to the old version of sniffer after going through 600 emails by hand and sorting out spam and real mail and manually placing them in the correct mailboxes, that was fun. Sorry about that. I tried to be as explicit as possible in the readme files and the help system in the program itself (running SNFClient.exe on the command line by itself should list all of the ways the client can be called.) The new SNF doesn't have a persistent and non-persistent mode like the old version. Instead, it is strictly a client/server model. Run the SNFServer.exe program as described in the read me and then leave it running. Then, with your message processing script you can call the SNFClient.exe program in place of the old SNF program without any modifications if that is easier -- -the SNFClient.exe will accept the same parameters as the old SNF program without a problem. This makes it relatively easy to switch back (as you did). If you start out running the SNFServer from the command line then it's display will help you to know when things are working correctly -- you will be able to see when messages start going through and you should quickly get an idea of what looks correct. Once you're confident in that setup then you can run the SNFServer using srvany or firedaemon or your other favorite utility that runs programs as a service. Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Error Messages since WeightGate
Darrell, Did you alter your heap size 3rd entry? If so, did you go to 1024 or other. I found this article by crossing a Declude page, appears to be what I need to go after. http://support.microsoft.com/default.aspx?scid=kb;EN-US;q142676 -Keith _ From: Message Sniffer Community on behalf of Darrell ([EMAIL PROTECTED]) Sent: Sun 6/10/2007 2:31 PM To: Message Sniffer Community Subject: [sniffer] Re: Error Messages since WeightGate After looking into it I am on board with what Pete said about the heap issue. It makes sense to me that its the heap issue since were launching weight gate - SNF. Effectively doubling the amount of processes being launched. Darrell --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Keith Johnson wrote: Darrell, You are right, a reboot will take care of it for a season, then it comes back out of the blue. Very strange indeed. Keith _ From: Message Sniffer Community on behalf of Darrell ([EMAIL PROTECTED]) Sent: Sat 6/9/2007 9:36 PM To: Message Sniffer Community Subject: [sniffer] Re: Error Messages since WeightGate Keith, I was having the same problems last week. Just came out of the blue and was across several of our servers as well. Same error verbatim. FWIW - I also use weightgate. I rebooted the servers I was seeing this issue on and the problem has not returned. Very odd you mentioned that as I thought this was isolated to just me. Darrell --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Keith Johnson wrote: It appears since installing WeightGate we have been receiving a lot of the below Application PopUps indicating an error: The application failed to initialize properly 0xc142. Click on OK to terminate the application The application entry is our Sniffer .exe. Today alone I saw over 300. I thought it was an isolated issue. However, it is happening across all our servers. We are running the latest Sniffer in Persistent mode. We never saw these prior to WeightGate. Has anyone seen this before? Below is the actual entry in Event Log. -Keith Event Type: Information Event Source: Application Popup Event Category: None Event ID: 26 Date: 6/9/2007 Time: 12:12:35 AM User: N/A Computer: NAIMAIL2 Description: Application popup: rrctp2ez.exe - Application Error : The application failed to initialize properly (0xc142). Click on OK to terminate the application. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Error Messages since WeightGate
is using it to display these error messages. So until you click OK to all the hundreds of pop-up boxes, or you reboot the computer, new mail will not be delivered. Eventually, the server may crash completely. Additional Help Most versions of Windows NT/2000 will apparently by default allocate 512KB of the Mystery Heap to each service-started process. There is also apparently a total of 48MB of the Mystery Heap available. That means you can have a maximum of about 77 service-started processes (48Megs minus (3Meg * 3 default desktops) minus (1Meg system-wide) divided by 512). Changing it to 256KB should approximately double the amount of service-started processes that can run before the mystery heap is depleted. However, some people have reported better results by raising the value to 2048KB -- that's one of the problems with undocumented resources (there's no way to know for sure which value is better or why). We recommend going to http://support.microsoft.com/default.aspx?scid=kb;EN-US;q142676 and changing the registry entry to use a value of 256 or 2048 (NOTE: Microsoft recommends 512 in that article; if you use 512, make sure not to have IMail's MaxQueProc registry entry set to more than 30). Matt Keith Johnson wrote: Darrell, Did you alter your heap size 3rd entry? If so, did you go to 1024 or other. I found this article by crossing a Declude page, appears to be what I need to go after. http://support.microsoft.com/default.aspx?scid=kb;EN-US;q142676 -Keith _ From: Message Sniffer Community on behalf of Darrell ([EMAIL PROTECTED]mailto:[EMAIL PROTECTED]) Sent: Sun 6/10/2007 2:31 PM To: Message Sniffer Community Subject: [sniffer] Re: Error Messages since WeightGate After looking into it I am on board with what Pete said about the heap issue. It makes sense to me that its the heap issue since were launching weight gate - SNF. Effectively doubling the amount of processes being launched. Darrell --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Keith Johnson wrote: Darrell, You are right, a reboot will take care of it for a season, then it comes back out of the blue. Very strange indeed. Keith _ From: Message Sniffer Community on behalf of Darrell ([EMAIL PROTECTED]mailto:[EMAIL PROTECTED]) Sent: Sat 6/9/2007 9:36 PM To: Message Sniffer Community Subject: [sniffer] Re: Error Messages since WeightGate Keith, I was having the same problems last week. Just came out of the blue and was across several of our servers as well. Same error verbatim. FWIW - I also use weightgate. I rebooted the servers I was seeing this issue on and the problem has not returned. Very odd you mentioned that as I thought this was isolated to just me. Darrell --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Keith Johnson wrote: It appears since installing WeightGate we have been receiving a lot of the below Application PopUps indicating an error: The application failed to initialize properly 0xc142. Click on OK to terminate the application The application entry is our Sniffer .exe. Today alone I saw over 300. I thought it was an isolated issue. However, it is happening across all our servers. We are running the latest Sniffer in Persistent mode. We never saw these prior to WeightGate. Has anyone seen this before? Below is the actual entry in Event Log. -Keith Event Type: Information Event Source: Application Popup Event Category: None Event ID: 26 Date: 6/9/2007 Time: 12:12:35 AM User: N/A Computer: NAIMAIL2 Description: Application popup: rrctp2ez.exe - Application Error : The application failed to initialize properly (0xc142). Click on OK to terminate the application. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.commailto:sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED]mailto:[EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]mailto:[EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]mailto:[EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]mailto:[EMAIL PROTECTED] -- --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. # This message is sent to you because you are subscribed to the mailing list sniffer
[sniffer] Re: Configuring Sniffer in declude....
Pete, If you don't mind, does WeightGate add any noticeable CPU cycles to run on top of running Sniffer? Thanks for the aid. Keith Johnson -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, November 29, 2006 4:57 PM To: Message Sniffer Community Subject: [sniffer] Re: Configuring Sniffer in declude Hello Chuck, If I might jump in here -- you are basically correct but you'll have to rename ShowMe.exe to the original weightgate name. When it is named ShowMe.exe it only records the command line parameters in a log file as a debugging aid. Second, with that done this should work fine as long as each command line is identical in Declude. Third, I noticed that your group IDs are out of date (based on the names you've used) and most likely you will want to revisit your weights also. A reference to the current group IDs (result codes) can be found here: http://kb.armresearch.com/index.php?title=Message_Sniffer.TechnicalDetai ls.ResultCodes Hope this helps, _M Wednesday, November 29, 2006, 3:48:21 PM, you wrote: Darrell: If I want to use Weightgate I assume that I put it in for each instance of sniffer. Such as - SNF external 063 c:\tool\ShowMe.exe -50 %WEIGHT% 30 c:\SNF\sniffer.exe authenticationxx 10 0 Chuck Schick Warp 8, Inc. (303)-421-5140 www.warp8.com -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Darrell ([EMAIL PROTECTED]) Sent: Wednesday, November 29, 2006 12:33 PM To: Message Sniffer Community Subject: [sniffer] Re: Configuring Sniffer in declude Chuck, Declude will only call Sniffer one time as long as the path and executable are identical which they are. Darrell -- -- Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. - Original Message - From: Chuck Schick [EMAIL PROTECTED] To: Message Sniffer Community sniffer@sortmonster.com Sent: Wednesday, November 29, 2006 2:16 PM Subject: [sniffer] Configuring Sniffer in declude Several years ago when we first started using message sniffer I set it up for in the following manner in my global.cfg file. SNIFFER-GENERALexternal063 F:\IMail\Declude\sniffer2r32\licensecode.exe activationcode 7 SNIFFER-EXPERIMENTALexternal062 F:\IMail\Declude\sniffer2r32\licensecode.exe activationcode 12 0 SNIFFER-OBFUSCATIONexternal061 F:\IMail\Declude\sniffer2r32\licensecode.exe activationcode11 So one and so forth. With the increase in spam and CPU load is there any advantage load wise to just call sniffer once using nonzero instead of the return code. It seems like someone told me that sniffer was only called once and not seperately for each return code. Could someone confirm that. Chuck Schick Warp 8, Inc. (303)-421-5140 www.warp8.com # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL
RE: [sniffer] Forwarding Spam
Pete, Perfect. I will work on setting up this account for you. What email address can I use to send over the POP info? Keith From: [EMAIL PROTECTED] on behalf of Pete McNeil Sent: Tue 9/6/2005 11:00 PM To: Keith Johnson Subject: Re: [sniffer] Forwarding Spam On Tuesday, September 6, 2005, 10:12:31 PM, Keith wrote: KJ Pete, KJ In the past, we have been processing missed spam KJ ourselves by asking our end customers, those who can, forward us KJ the header and we will adjust our filters. However, I notice on KJ the sortmonster.com website that one can forward to an address KJ missed spam and Sniffer can make adjustments. Can you take the KJ message in its simply forwarded state or do you need the header? KJ What email address should we be sending these to? This would KJ reduce the amount of processing on our side if we could automate KJ this process. Thanks again for the aid. The best way to do this is to set up an address on your system that we can access with POP. We will program our robots to get the messages and place them in our processing queues. We have two classifications for these -- The first is clean spamtraps -- that is, messages that were sent to invalid and harvested addresses. The second are usertraps --- these are messages that are either submitted by users as spam or come from otherwise impure spamtraps (such as long dead accounts etc...) In both cases we like the message as close to it's original form as possible --- but simple forwards work just fine. When we process usertrap messages we consider them anonymously and with a good deal of suspicion. I don't want to give you the impression that we create rules for your system based on these kinds of submissions, but we do create rules for the core rulebase if they are appropriate... To the extent that some spam reaches your system before it would otherwise reach us, your submissions would help everyone by accelerating the speed by which we add new rules for new campaigns. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html winmail.dat
[sniffer] Forwarding Spam
Pete, In the past, we have been processing missed spam ourselves by asking our end customers, those who can, forward us the header and we will adjust our filters. However, I notice on the sortmonster.com website that one can forward to an address missed spam and Sniffer can make adjustments. Can you take the message in its simply forwarded state or do you need the header? What email address should we be sending these to? This would reduce the amount of processing on our side if we could automate this process. Thanks again for the aid. Keith winmail.dat
RE: [sniffer] false positives which catagories?
Scott, HS = Test says ham, final result was spam. This is an inaccurate ham result. 'False negative' How are you auto determining that an email that was ham was really spam? Are you keying in this info into your stats based on your viewing of the email or by user complaint? Obviously, if Declude triggers and email to have action on it based on spam settings it was spam and if it didn't take action on it and it went through to your users it was ham. Thanks again for the aid. Keith From: [EMAIL PROTECTED] on behalf of Scott Fisher Sent: Thu 8/4/2005 10:02 AM To: sniffer@SortMonster.com Subject: Re: [sniffer] false positives which catagories? I have my sniffer result histories by category posted at: http://it.farmprogress.com/declude/Testsbymonth.html Look about 90% down the page. - Original Message - From: Bonno Bloksma mailto:[EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Thursday, August 04, 2005 1:40 AM Subject: [sniffer] false positives which catagories? Hi, I'd like to make a difference in the ways I score the varions sniffer catagories in Declude. I hold at 20 and have had the several sniffer catagories all at 19. As we are a school for tourism I score sniffer travel lower but I would like to score some catagories higher, at 20. If we have a false positive it's mostly in the general, exp-abstract, ip-rules catagorie is my feeling. Someone must have made a comparison of false positives against sniffer and in which catagories those fp's are mostly. Right? Which catagories have virtually no FPs and which should I keep (well) below my hold level? Of course all held mail gets reviewed by be, unless it scrores enough other points te get deleted (at 27 points). Groetjes, Bonno Bloksma winmail.dat
RE: Re[2]: [sniffer] Persistent Sniffer
Pete, Thanks for the reply. Running on an IBM Xseries 225 Dual Xeon 2.4Ghz w/ 1GB RAM - running IBM's ServerRAID 5i in IBM's RAID 10 config (4 73GB 10K drives) - O/S is Windows 2000 Standard Server SP4 Running Imail 8.15HF1 with Declude JM/Virus 1.82 - BIND DNS Server is 1 hop away (on switch backbone). I had to drop back to the non-persistent mode, thus the .stat file disappeared. I will run it again tonight and copy the file away and post it here tonight. Thanks again for the time and aid. Keith Johnson -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Friday, April 01, 2005 11:17 AM To: Keith Johnson Subject: Re[2]: [sniffer] Persistent Sniffer On Friday, April 1, 2005, 8:04:27 AM, Keith wrote: KJ I have read forum results that this behavior is the reverse of what KJ should happen, I should get a reduction in CPU. I did this around KJ 11pm last night, usually during peak times this server would stay at KJ 65% load. Is there anything I can tweak to install the Sniffer KJ persistent server and achieve desired results? Thanks for the aid. Can you share more about your server's configuration and can you also post the .stat file that was produced? Server OS? Server CPU(s)? Drive System(s)? Mail Server SW? _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Persistent Sniffer
Pete, Wow, thank you for the explanation. I did let the persistent server run for 30 min after I restarted the services. However, I did stop the services, then started Sniffer service, then restart Imail services. I could have gotten a backlog of retries at that moment that pegged the CPU as you stated. We have batted around running BIND for NT/2000 on the local machine, but my fear was overhead of another major process running. I don't have any good stats on how much CPU/Memory BIND on an Imail Server requires, thus, we have a SUN/BIND box local to the switch. Are you aware of any stats on this? We don't run the AVAFTERJM switch. This is done in part due to so many of our customers still look at their spam email from time to time. We heavily use the ROUTETO and MAILBOX command, thus, if I let a virus go through to their to mailbox, they could potentially open a virus spam email and hurt themselves. We defrag each partition every night using Diskeeper and it works great. I regularly look at the Sniffer directory to ensure no left over .fin files and others that could cause server load. I will retry it again tonight and see what type of results I get and post them here. It could be as you say, I am on the far side :) Thanks again, Keith -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Friday, April 01, 2005 2:16 PM To: Keith Johnson Subject: Re[4]: [sniffer] Persistent Sniffer On Friday, April 1, 2005, 11:44:07 AM, Keith wrote: KJ Pete, KJ Thanks for the reply. KJ Running on an IBM Xseries 225 Dual Xeon 2.4Ghz w/ 1GB RAM - KJ running IBM's ServerRAID 5i in IBM's RAID 10 config (4 73GB 10K KJ drives) KJ - O/S is Windows 2000 Standard Server SP4 KJ Running Imail 8.15HF1 with Declude JM/Virus 1.82 - BIND DNS KJ Server is 1 hop away (on switch backbone). I had to drop back to KJ the non-persistent mode, thus the .stat file disappeared. I will KJ run it again tonight and copy the file away and post it here tonight. KJ Thanks again for the time and aid. I don't see any problems with this setup. Your description sounds like your server is fairly heavily loaded (35-55% cpu in peer-server mode), though I would expect more from the hardware you've described. I suspect that you may have run into the far side of the power curve when you went to persistent server mode. In peer-server mode the failure mode for overload conditions is much softer than with the persistent peer server mode. Up to the failure point in the power curve the persistent server mode will provide a significant savings over peer-server, however once that point is reached the persistent server mode tends to degrade much more quickly and requires a significant drop in load before recovery occurs. I'm working on some strategies to soften that curve a bit, but in the mean time let's explore these options to get the best performance from your server and reduce it's load. The we can see if the persistent server engine will give you even more headroom: 1. I recommend running AVAFTERJM - are you doing this? Typically 80% or more of email traffic is spam and so there is no good reason to attempt a virus scan on these messages. If you hold messages and occasionally re-insert them into the queue then they will not be scanned, however there are ways to work around this when needed - and it is very likely you would not re-insert a message that contained a virus anyway. 2. Consider running bind as a dns resolver on your mail server and pointing the server to itself via the loopback address (127.0.0.1) for DNS services. This tends to speed up processing significantly which also reduces the number of message processes that are running at any given time. YMMV, but I have seen this work consistently to improve performance. --- when trying persistent mode (minor adjustments really) --- A. Set the Persistence value in your snflicid.cfg file to 3600. - no need to check for a new rulebase every 10 minutes usually. These loop events tear down the server momentarily which can perturb an otherwise smooth running system when under heavy loads - thus minimizing the frequency of these events may help. B. Set LogFormat in your snflicid.cfg file to SingleLine. This provides sufficient data for our purposes (most of the time) and should significantly reduce the size of your log file. C. Be sure to keep any unnecessary files out of the SNF working directory - in particular you should clean out any orphaned files that might still be lurking from previous crashes. --- General --- Be sure your drives are regularly defragmented. Hope this helps, _M PS: I just had another random thought really --- Could it be that the high CPU value was appropriate? If you had built up a queue of messages to be processed then once the persistent server was put in place and the system started processing messages again the CPU would
RE: Re[8]: [sniffer] Persistent Sniffer
Pete, Yes the file is changing every few seconds or sooner. Sorry, I just did a 'grab' of it and posted. The 307 is due to me stopping it after 30 min or so and altering the few changes to the .conf file. I will continue to monitor it over the weekend. However, so far so good. Thanks again for taking the time to help out. Keith -Original Message- From: [EMAIL PROTECTED] on behalf of Pete McNeil Sent: Fri 4/1/2005 10:18 PM To: Keith Johnson Cc: Subject: Re[8]: [sniffer] Persistent Sniffer On Friday, April 1, 2005, 9:36:05 PM, Keith wrote: KJ Pete/Matt/Andrew, KJ Thanks for all your wonderful input. Maybe I didn't KJ give it a fair shake or time enough as mentioned by Pete earlier. KJ I turned it on again about 30 min ago and have seen my system KJ stable, currently it is: KJTicToc: 1112391330 KJ Loop: 264 KJ Poll: 445 KJ Jobs: 290 KJ Secs: 307 KJ Msg/Min: 56.6775 KJ Current-Load: 21.4724 KJ Average-Load: 22.4706 KJ These numbers were up around 120 Msg/Min and Current KJ Load at 90+CPU is aver. about 17% right now. However, could KJ be skewed a bit since it is Friday night. I will continue to KJ watch it over the weekend and see how it goes. Still considering KJ running Win DNS local or BIND 9.3 for NT/2000/2003. Have a great KJ weekend. Hrmmm Something here doesn't add up. Is the .stat file changing every second or so? If not then the persistent engine has stopped. In fact, 307 seconds is scarcely 5 minutes - not 30. It appears that at the time you sampled the file your system was happily humming along at about 1 msg/sec... which is a lul for you. Remember that your average would be about 1.7 messages per second. I also note that the load and poll time indicated a good deal of dead air so the system was definitely not working hard at the time. Take a look at it again and make sure that the .stat file changes every 1-4 seconds or so. If not then the persistent server has stopped - at least the client instances will see it that way. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html winmail.dat
[sniffer] Persistent Sniffer
I noticed in the archives about a .cfg file one can configure for use when running Persistent sniffer. How do you download it or obtain it? Thanks for the aid. Keith This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Determine Version
Is there a easy way to determine the Sniffer version you are running (i.e. command line or the like)? Thanks for the aid. Keith winmail.dat
RE: [sniffer] New Version 2-3.2 has been officially released.
We run Sniffer in the normal way (non-persistent), is there an extra file that we must copy into the Sniffer directory in order for this version to work properly? I believe I read somewhere of a config file that contains needed settings. Thanks again, Keith -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Tuesday, November 23, 2004 2:58 AM To: [EMAIL PROTECTED] Subject: [sniffer] New Version 2-3.2 has been officially released. Hello Sniffer Folks, We have now officially released version 2-3.2 of Message Sniffer. You can download the distribution files from our Try-It page. This version includes a number of upgrades that will improve the spam filtering performance of Message Sniffer by allowing it to see beyond most obfuscation mechanisms. In particular, this version makes obfuscation techniques that use HTML and XML tags, HTML encoding, and URL encoding ineffective in most cases. These new features do not interfere with Message Sniffer's ability to detect these obfuscation techniques, but rather enhances these capabilities to allow clear-text patterns to match obfuscated message content in addition to any other detection rules that might apply. (Version 2-3.2 is functionally identical to version 2-3.1i2 which has successfully passed internal and external testing.) This is an important upgrade. As we begin to generate rules that take advantage of these new features, any systems that are running the older version may experience a decrease in performance over time. This version is a drop-in replacement for version 2-3.1. This version is compatible with the prior 2.x versions. You may install the new .exe by renaming it for your license ID and replacing your current .exe file. (You will probably need to temporarily stop your email server software and any persistent instance of Message Sniffer before you can replace the .exe file on your system.) Thanks, _M Pete McNeil (Madscientist) President, MicroNeil Research Corporation Chief SortMonster (www.sortmonster.com) This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] New Version 2-3.2 has been officially released.
Pete, We plan to, working on the SrvAny service in beta right now. Thanks again for the aid and time. Keith -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Tuesday, November 23, 2004 5:07 PM To: Keith Johnson Subject: Re[2]: [sniffer] New Version 2-3.2 has been officially released. On Tuesday, November 23, 2004, 4:19:35 PM, Keith wrote: KJ We run Sniffer in the normal way (non-persistent), is there an extra KJ file that we must copy into the Sniffer directory in order for this KJ version to work properly? I believe I read somewhere of a config KJ file that contains needed settings. Thanks again, Nothing in the .cfg file is strictly needed. If you don't have one, then copy the one that comes with the distribution. You _may_ want to use one or more of the features at some point. If you don't have it then it is ignored. (Backward compatibility). _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Your Sniffer Setup
Thanks Andy and Bill, will give this a go on our beta server. Thanks again for the time and expertise Keith -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andy Schmidt Sent: Monday, November 01, 2004 12:11 PM To: [EMAIL PROTECTED] Subject: RE: [sniffer] Your Sniffer Setup Hi Landry: These simplified instructions only apply if the application needs no parameters, as it only covers the application key: Value Name: Application Data Type : REG_SZ String : path\application.ext If there was a SnifferPersistent.exe that needed no further options, these simplified instructions would work For Sniffer however, you (supposedly) do need to pass along the authorizaton code and the persistent option, which are defined in the AppParameters value in the registry. That's how the previous version worked for me. Immediately upon upgrading to the latest version, Sniffer would no longer find its directory when executed as a service, so I had to add the AppDirectory key to set the working directory. Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Landry William Sent: Monday, November 01, 2004 11:03 AM To: '[EMAIL PROTECTED]' Subject: RE: [sniffer] Your Sniffer Setup See http://support.microsoft.com/default.aspx?scid=kb;en-us;137890 for simplified instructions. Bill This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Version 2-3.0i8 published.
If we don't run the Mdaemon on our systems and just use the new download, will we also see a speed increase on processing. Thanks for the time. Keith -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, October 20, 2004 1:50 PM To: Frank Osako Subject: Re[4]: [sniffer] Version 2-3.0i8 published. On Wednesday, October 20, 2004, 12:54:04 PM, Frank wrote: FO Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog FO See a reduction of what in their backlog? Can you give an example FO of how to see this type of measurement? Another good question - I will try to get a solid, detailed answer. I'm not an MDaemon expert so I'm not sure what the best strategies are for measuring throughput performance and backlog (inbound/outbound queue length). Perhaps there are some MDaemon experts on list that can share their strategies for making these measurements? In particular, how best to measure these things when the system in question is not overloaded? Thanks, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Version 2-3.0i2 release.
Pete, I take it this can be run without the persistent mode? Thanks for the aid. Keith -Original Message- From: [EMAIL PROTECTED] on behalf of Pete McNeil Sent: Mon 9/13/2004 9:15 PM To: [EMAIL PROTECTED] Cc: Subject: [sniffer] Version 2-3.0i2 release. Hello Sniffer Folks, Please find interim update 2 at the following link: http://www.sortmonster.com/MessageSniffer/Betas/MessageSniffer2-3.0i2-Distribution.zip This distribution patches a hole in the FilterChain module of the scanner. In prior versions it was possible for the unexpected presence of a 'null' character to prevent the remainder of a message from being scanned. In theory (not yet proven) this could cause some rules not to fire on a message even though the rules would be present in the rule base. This is a minor adjustment which has tested well on our servers. We will be making this the official distribution after a little more testing. No problems have been observed or reported so far. Thanks, _M Pete McNeil (Madscientist) President, MicroNeil Research Corporation Chief SortMonster (www.sortmonster.com) This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html winmail.dat
[sniffer] FIN File
I found a .fin file in my sniffer directory and didn't know if anyone knew what it was and how it is produced. It is dated several days ago. Thanks for the aid. Keith This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Rules Question
I am using Declude and have indiv. Sniffer Tests and lets say the following gets tripped in an email SNIFFER-WHTLIST result code 000 SNIFFER-PORNresult code 054 Which would take precedence over the other, as far as which would be the final code passed to Declude? Thanks, Keith This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Rules Question
Thanks for the aid. One last question, you mentioned: In a case where a white rule is present and a black rule is present the white rule will always win So if the White Rule fired 000, it would override a Porn Rule of 54? If so, how are these White Rules entered? Thanks, Keith -Original Message- From: [EMAIL PROTECTED] on behalf of Madscientist Sent: Wed 3/3/2004 6:01 PM To: [EMAIL PROTECTED] Cc: Subject: Re: [sniffer] Rules Question At 04:55 PM 3/3/2004, you wrote: I am using Declude and have indiv. Sniffer Tests and lets say the following gets tripped in an email SNIFFER-WHTLIST result code 000 SNIFFER-PORNresult code 054 Which would take precedence over the other, as far as which would be the final code passed to Declude? There is some confusion about this. A zero result from Message Sniffer as seen by Declude could mean that a white rule has fired, or it could mean that no rules matched at all. In the first case - where an actual white rule has fired, the Message Sniffer log will show a White entry and the Final result will reflect that white rule. In this case, the white rule takes precedence. Declude will see a 0 result code. In the second case - where no rules matched, the Message Sniffer log will show a Clean entry and Declude will see a zero result. So, from Declude's perspective it will see a zero result in both the Clean and the White case. As a result, your SNIFFER-WHTLIST result code 000 test will fire. In a case where a white rule is present and a black rule is present the white rule will always win. So, if Sniffer saw both rules match a message it would return a zero result. SNIFFER-WHTLIST is a misnomer. It's probably not a good idea to name the zero result test this way because most of the time a zero result doesn't mean White but instead means Clean. If you wish to have the white rules in your rulebase separated out then we could code those to a 1 result and then you would be able to legitimately create a SNIFFER-WHTLIST test checking for a result of 1. I will point out here that this has been tried once or twice and in both cases the user switched back almost immediately because the results were confusing. In Sniffer we use white rules to force a non result more than we ever use them to indicate a true white result. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html winmail.dat
