[sniffer] Sniffer 3.0 Installed
Hi, Didn't realize I had been uninstalled for a few months. I saw that V3 was released, so I gave it a shot. I unzipped the installation files to a new /SNF folder. All files were expanded into the same folder (your zip file had not subfolders!). Following the instructions I customized the XML files. I noticed THESE parameters: node identity='D:/IMail/declude/SNF/identity.xml' paths log path='D:/IMail/declude/SNF/Log/'/ rulebase path='D:/IMail/declude/SNF/Rulebase/'/ workspace path='D:/IMail/declude/SNF/Workspace/'/ /paths I'm a believer in keeping different data in their distinct subfolders, so I set up the /Log, /Rulebase and /Workspace subfolders by hand and updated the XML file. The I took a wild guess that SOME files would have to be moved into those subfolders - but there are NO instructions WHAT files go WHERE for things to actually work! I found it annoying that further down in the same XML File was yet another path that was NOT included in the paths node in the top of the XML file: update-script on-off='on' call='D:/IMail/declude/SNF/getRulebase.cmd' guard-time='180'/ Next I had to customize the getRuleBase.cmd - because it too does NOT support the use of the rulebase/workspace paths. Here was yet ANOTHER place where I had to manually configure the same path information again, as well as the license key. Needless to say, I'm not a friend of having redundant path information in several locations as this is an unnecessary source of error. Through testing I determined that some more files had to be moved to certain sub folders for things to work: UpdateReady.txt - /Workspace GBUdbIgnoreList.text - /Workspace Your .SNF - /Rulebase Then I had to further adapt the getRuleBase.cmd because throughout this procedure, you need to prefix references to the rulebase and the UpdateReady.* files with the appropriate paths for things to actually work. At this point, I'm still no clear where the mingwm10.dll, exchndl.dll and AuthenticationProtocol.swf need to reside! I didn't move them, but I'm not sure if that creates a problem down the road. Here are my suggestions: a) Snf_engine.xml should have one ApplPath parameter where I can just define 'D:/IMail/declude/SNF'. Unless I OVERRIDE any of the other paths, it should know the that by default the other paths are all assumed to be below the ApplPath and no extra parameters are necessary: identity.xml getRulebase.cmd Log/ Rulebase/ Workspace/ b) There should be a simple command line utility (e.g., SNFClient.exe -Paths) to automatically create Environment Variables for the paths. This way, this command can just be included at the beginning of the getRuleBase script and one doesn't have to manually hardcode those same paths into yet another location. PS: Here is my corrected version of the getRuleBase CMD file that looks for the files in the correct subfolders: @ECHO OFF SETLOCAL REM - Edit This Section SET SNIFFER_PATH=D:\IMail\declude\SNF SET RULEBASE_PATH=%SNIFFER_PATH%\Rulebase SET WORKSPACE_PATH=%SNIFFER_PATH%\Workspace SET AUTHENTICATION=authenticationxx SET LICENSE_ID=licenseid REM CD /d %SNIFFER_PATH% if not exist %WORKSPACE_PATH%\UpdateReady.txt GOTO DONE REM The next line may cause trouble if your system stops while this REM script is running. It is not needed when this script is run REM from SNF's update-script/ feature since only one copy will run REM at a time. However, if you are going to run a version of this REM script as a scheduled task you will want to uncomment the next REM line to make sure only one copy runs at a time-- just be sure to REM clean out any stale .lck files after a restart. REM if exist %WORKSPACE_PATH%\UpdateReady.lck GOTO DONE :DOWNLOAD COPY %WORKSPACE_PATH%\UpdateReady.txt %WORKSPACE_PATH%\UpdateReady.lck wget http://www.sortmonster.net/Sniffer/Updates/%LICENSE_ID%.snf -O %RULEBASE_PATH%\%LICENSE_ID%.new.gz --header=Accept-Encoding:gzip --http-user=sniffer --http-passwd=ki11sp8m if exist %RULEBASE_PATH%\%LICENSE_ID%.new.gz gzip -d -f %RULEBASE_PATH%\%LICENSE_ID%.new.gz snf2check.exe %RULEBASE_PATH%\%LICENSE_ID%.new %AUTHENTICATION% if errorlevel 1 goto CLEANUP if exist %RULEBASE_PATH%\%LICENSE_ID%.old del %RULEBASE_PATH%\%LICENSE_ID%.old rename %RULEBASE_PATH%\%LICENSE_ID%.snf %LICENSE_ID%.old rename %RULEBASE_PATH%\%LICENSE_ID%.new %LICENSE_ID%.snf if exist %WORKSPACE_PATH%\UpdateReady.txt del %WORKSPACE_PATH%\UpdateReady.txt if exist %WORKSPACE_PATH%\UpdateReady.lck del %WORKSPACE_PATH%\UpdateReady.lck :CLEANUP if exist %RULEBASE_PATH%\%LICENSE_ID%.new del %RULEBASE_PATH%\%LICENSE_ID%.new if exist %WORKSPACE_PATH%\UpdateReady.lck del %WORKSPACE_PATH%\UpdateReady.lck
[sniffer] Re: Sniffer 3.0 Installed
Hello Andy, First, let me say thanks for sharing all of this. We don't often get detailed feedback on these things. Your valuable insights will be used to make later releases better. With that said I will add a few comments here and there to explain why things are the way they are and help others achieve their goals (which might be different). Saturday, October 4, 2008, 2:12:47 AM, you wrote: Hi, Didnt realize I had been uninstalled for a few months. I saw that V3 was released, so I gave it a shot. I unzipped the installation files to a new /SNF folder. All files were expanded into the same folder (your zip file had not subfolders!). The vast majority of SNF installations on Windows systems keep all SNF components in the same folder. So, for the majority of folks SNF is simply decompressed into "it's own folder", configured, and launched. This makes things simple and there is no question where to find things. Along the way we have been asked for the ability to put logs in a different location, get rulebsae files from a different location, configuration files, and so forth. We've added those features so that the folks who have reason to move things around can do so. We decided not to create a presumed directory structure for SNF because the folks who've asked us to provide these features all had their own unique way to divide things and move them. Any structure we created would have been wrong for most folks, so we keep the single folder option as our default since it is what everyone was used to and what most of our customers have been using. SNF is used on a lot of platforms -- each with their own conventions. Not only that but within each platform administrators and user communities develop their own preferences. The paths/ section described next allows folks to manipulate some file locations according to user preferences. Following the instructions I customized the XML files. I noticed THESE parameters: node identity='D:/IMail/declude/SNF/identity.xml' paths log path='D:/IMail/declude/SNF/Log/'/ rulebase path='D:/IMail/declude/SNF/Rulebase/'/ workspace path='D:/IMail/declude/SNF/Workspace/'/ /paths Im a believer in keeping different data in their distinct subfolders, so I set up the /Log, /Rulebase and /Workspace subfolders by hand and updated the XML file. The I took a wild guess that SOME files would have to be moved into those subfolders but there are NO instructions WHAT files go WHERE for things to actually work! The current documentation is located here: http://www.armresearch.com/support/articles/software/snfServer/config/node/paths/index.jsp The general design is such that log files will be written into the log path, the rulebase file will be read from the rulebase path, and the remaining files should reside in the workspace path. I will add a task to clarify this in our documentation and provide more detail. I found it annoying that further down in the same XML File was yet another path that was NOT included in the paths node in the top of the XML file: update-script on-off='on' call='D:/IMail/declude/SNF/getRulebase.cmd' guard-time='180'/ The configuration file is organized by function. The top of the configuration file and in particular the paths/ section is concerned with describing the architecture of the SNFServer installation. The update-script/ feature is a component of the networking section because it is triggered by SNF network operations, so we put it's configuration information in that section. This feature is still evolving -- in it's original design it was presumed that the update script would reside in the single SNF directory, or perhaps in the workspace directory -- so only the name of the script would be required in this location. We actually have had quite a few successful installations this way. However, along the way we've determined that the update script might be located anywhere on the system and that we could not always assume the current workspace for SNFServer indicated the location (or even a relative location) for the udpate script. To prevent errors we've taken to coding the full path to the script in this section of the configuration. Another part of the thinking on this is that the update-script feature is completely optional. In fact many of the larger systems that we service use entirely separate update mechanisms and turn this feature off. It seemed to make more sense to put the script path closer to the network features that trigger it. Next I had to customize the getRuleBase.cmd because it too does NOT support the use of the rulebase/workspace paths. Here was yet ANOTHER place where I had to manually configure the same path information again, as well as the license key. Needless to say, Im not a friend of having redundant path information in several locations as this is an unnecessary source of error. This is an unfortunate, but necessary
[sniffer] Re: Sniffer 3.0 Froze Mail Server
Ouch - 3.0 didn't even last 12 hours. Imail was frozen up because it apparently couldn't launch any more Sniffer client instances. Event Log was full with: Event Type:Information Event Source:Application Popup Event ID: 26 Description: Application popup: SNFClient.exe - Application Error : The application failed to initialize properly (0xc142). Click on OK to terminate the application. Had to manually kill a HUGE multiple list of imailsrv.exe's (taskkill /im imailsrv.exe /f ) and a similar long list of SNFClient.exe's. Normally, this Imail Server runs unattended for weeks until a Windows security update requires reboot! Best Regards, Andy From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Andy Schmidt Sent: Saturday, October 04, 2008 2:13 AM To: Message Sniffer Community Subject: [sniffer] Sniffer 3.0 Installed Hi, Didn't realize I had been uninstalled for a few months. I saw that V3 was released, so I gave it a shot. I unzipped the installation files to a new /SNF folder. All files were expanded into the same folder (your zip file had not subfolders!). Following the instructions I customized the XML files. I noticed THESE parameters: node identity='D:/IMail/declude/SNF/identity.xml' paths log path='D:/IMail/declude/SNF/Log/'/ rulebase path='D:/IMail/declude/SNF/Rulebase/'/ workspace path='D:/IMail/declude/SNF/Workspace/'/ /paths I'm a believer in keeping different data in their distinct subfolders, so I set up the /Log, /Rulebase and /Workspace subfolders by hand and updated the XML file. The I took a wild guess that SOME files would have to be moved into those subfolders - but there are NO instructions WHAT files go WHERE for things to actually work! I found it annoying that further down in the same XML File was yet another path that was NOT included in the paths node in the top of the XML file: update-script on-off='on' call='D:/IMail/declude/SNF/getRulebase.cmd' guard-time='180'/ Next I had to customize the getRuleBase.cmd - because it too does NOT support the use of the rulebase/workspace paths. Here was yet ANOTHER place where I had to manually configure the same path information again, as well as the license key. Needless to say, I'm not a friend of having redundant path information in several locations as this is an unnecessary source of error. Through testing I determined that some more files had to be moved to certain sub folders for things to work: UpdateReady.txt - /Workspace GBUdbIgnoreList.text - /Workspace Your .SNF - /Rulebase Then I had to further adapt the getRuleBase.cmd because throughout this procedure, you need to prefix references to the rulebase and the UpdateReady.* files with the appropriate paths for things to actually work. At this point, I'm still no clear where the mingwm10.dll, exchndl.dll and AuthenticationProtocol.swf need to reside! I didn't move them, but I'm not sure if that creates a problem down the road. Here are my suggestions: a) Snf_engine.xml should have one ApplPath parameter where I can just define 'D:/IMail/declude/SNF'. Unless I OVERRIDE any of the other paths, it should know the that by default the other paths are all assumed to be below the ApplPath and no extra parameters are necessary: identity.xml getRulebase.cmd Log/ Rulebase/ Workspace/ b) There should be a simple command line utility (e.g., SNFClient.exe -Paths) to automatically create Environment Variables for the paths. This way, this command can just be included at the beginning of the getRuleBase script and one doesn't have to manually hardcode those same paths into yet another location. PS: Here is my corrected version of the getRuleBase CMD file that looks for the files in the correct subfolders: @ECHO OFF SETLOCAL REM - Edit This Section SET SNIFFER_PATH=D:\IMail\declude\SNF SET RULEBASE_PATH=%SNIFFER_PATH%\Rulebase SET WORKSPACE_PATH=%SNIFFER_PATH%\Workspace SET AUTHENTICATION=authenticationxx SET LICENSE_ID=licenseid REM CD /d %SNIFFER_PATH% if not exist %WORKSPACE_PATH%\UpdateReady.txt GOTO DONE REM The next line may cause trouble if your system stops while this REM script is running. It is not needed when this script is run REM from SNF's update-script/ feature since only one copy will run REM at a time. However, if you are going to run a version of this REM script as a scheduled task you will want to uncomment the next REM line to make sure only one copy runs at a time-- just be sure to REM clean out any stale .lck files after a restart. REM if exist %WORKSPACE_PATH%\UpdateReady.lck GOTO DONE :DOWNLOAD COPY %WORKSPACE_PATH%\UpdateReady.txt %WORKSPACE_PATH%\UpdateReady.lck wget
[sniffer] Re: Sniffer 3.0 Installed
Hello Andy, Saturday, October 4, 2008, 12:28:44 PM, you wrote: HI Pete, Thanks for your feedback. I had to create the UpdateReady.txt file before I was able to test my update script from the command line but I didnt realize that I would be created in the Workspace folder. Without that information, one cannot adapt the update script to ones needs. Since the server always creates UpdateReady file in the Workspace folder and always expects the .SNF file in the Rulebase folder, its pretty safe to say that anyone using the getRuleBase.cmd would absolutely have to add the Workspace and Rulebase paths otherwise they cant possibly find the UpdateReady file and the script will just exit OR it will not place the SNF File where the server will find it. Anyone who has their own update mechanism clearly doesnt fall under this discussion at all. My conclusion is, that the current getRuleBase.cmd only handles the case when there are no separate directories but the with the changes I made, the getRuleBase.cmd would allow a user to define separate directories at the top of the script (if thats how they configured things) and thus correctly handle a SINGLE as well as separate directories. In my opinion, that is the more correct behavior. I'm still trying to think of a way to describe this modification so that it makes sense without causing lots of confusion. Since we're trying to reach a larger audience these days we've created a generalized approach and built an installer that configures SNF, Declude, mxGuard, and MINIMI on IMail, SmarterMail, and even a "generic" (roll your own) configuration. The installer performs upgrades from the previous version as well. The idea is that more folks won't have to do any tweaking at all. Your proposed getRulebase works great for your structure-- and I agree it's a nice idea to have a WORKSPACE_PATH and RULEBASE_PATH variable.. BUT I'm having a hard time figuring out a way to include those and their various options without adding a lot of confusion and complexity... The existing getRulebase script works perfectly when used with the installer and nobody has to touch it. My best thinking at the moment is to perhaps do something like this: REM - Edit This Section - SET LICENSE_ID=licenseid SET AUTHENTICATION=authenticationxx SET SNIFFER_PATH=D:\IMail\declude\SNF REM Modify the next two lines if you modify SNF's directory structure. SET RULEBASE_PATH=%SNIFFER_PATH% SET WORKSPACE_PATH=%SNIFFER_PATH% REM - Of course doing that would mean rewriting our installer too (Since it needs to modify/generate the getRulebase script. For the immediate future this discussion is archived and searchable and I will add a task to the web site project to describe some of these getRulebase.cmd scenarios. How does that sound? _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Sniffer 3.0 Installed
Hi Pete, My best thinking at the moment is to perhaps do something like this Right, exactly. As long as the parameters are already there to be modified and the script uses those parameters, then the script is ready to go for any user (with or without distinct directories). Of course doing that would mean rewriting our installer too (Since it needs to modify/generate the getRulebase script. Yes, if you want the installer to handle the subdirectory layout, then it would have to adapt the additional two lines in the getRulesbase script - which would make it more flexible to deal with different customer scenarios. Best Regards, Andy From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Saturday, October 04, 2008 3:52 PM To: Message Sniffer Community Subject: [sniffer] Re: Sniffer 3.0 Installed My best thinking at the moment is to perhaps do something like this: REM - Edit This Section - SET LICENSE_ID=licenseid SET AUTHENTICATION=authenticationxx SET SNIFFER_PATH=D:\IMail\declude\SNF REM Modify the next two lines if you modify SNF's directory structure. SET RULEBASE_PATH=%SNIFFER_PATH% SET WORKSPACE_PATH=%SNIFFER_PATH% REM - Of course doing that would mean rewriting our installer too (Since it needs to modify/generate the getRulebase script. For the immediate future this discussion is archived and searchable and I will add a task to the web site project to describe some of these getRulebase.cmd scenarios. How does that sound? _M
[sniffer] Re: FW: [sniffer] Re: Sniffer 3.0 Froze Mail Server
Hi Pete, Well, I eliminated WeightGate for the time being, just to do my due diligence. Also, since there is a fix sized buffer, I assume actually LOWERING the 3rd number (the allocation for each non-interactive process) would allow for MORE parallel processes to run (as long as the value is still large enough to support each of the applications that rely on it.) Of course, I assume the heap issue in reality is actually a SECONDARY problem ( a symptom of too many non-interactive tasks being launched and not completing). Since the 'heap' space is finite, there is a hard limit as to how many processes can be in a wait state at the same time. The problem to focus on is not the known, limited heap, but rather the reason why these processes were unable to complete and thus eventually too many processes being active. Best Regards, Andy From: Pete McNeil [mailto:[EMAIL PROTECTED] Sent: Saturday, October 04, 2008 10:07 PM To: Andy Schmidt Cc: [EMAIL PROTECTED] Subject: Re: FW: [sniffer] Re: Sniffer 3.0 Froze Mail Server Hello Andy, Saturday, October 4, 2008, 9:22:39 PM, you wrote: Hi Pete, Here the log files. I can't tell you WHEN the problem was triggered. I was off site and was alerted around noon that the SMTP service had become unresponsive. I assumed it had crashed, but found it running. Thus I tried restarting the SMTP service, but after shutting down, it wouldn't allow me to restart. That's when I started looking a bit more closely. Once I realized that I had all these SNFclient processes running (I checked the event log to see if it would give me any clue - but since the errors had been occurring for a while, my system event log had wrapped around, so I couldn't tell when it actually started and how long it may have taken between the actual problem and until the SMTP service became unresponsive. This Imail server is a PowerEdge 2950, Quad CPU, 3GHz. 2 GB of RAM and normally using about 1.5 GB of virtual RAM and on weekends, CPU load is usually below 10%. When this was going on, I didn't pay close attention because I wasn't quite sure yet what was going on and was trying to figure out how to get out of it. But, based on the memory use graph, I would guess it had maxed out 4 GB of virtual RAM, which eventually starved the SMTP service and prevented it from accepting more connections.. As soon as I flushed the command line programs, the memory curve dropped very sharply by easily half. Sorry - don't have anything more specific. I've been watching your telemetry and I don't think the problem was triggered by an ordinary overload. Your message rate is not high enough to cause that -- SNFClients will only wait about 30 seconds or so at most if they are unable to make contact - - even on the busiest systems. The other thing that strikes me is that you had to kill a lot of imailsrv.exe instances as well-- this is new and very different. Once the mystery heap was exhausted I would expect SNFClient instances to build up in a broken state (0x142) but there is no good reason for imailsrv instances to build up that I can think of -- except maybe some kind of list processing event? (IIRC, imailsrv is called to handle list processing requests through an alias -- it's been a while). I will check the SNF log to see if I can identify anything useful. Thanks, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC.
[sniffer] Re: FW: [sniffer] Re: Sniffer 3.0 Froze Mail Server
Hello Andy, Saturday, October 4, 2008, 10:21:31 PM, you wrote: Hi Pete, Well, I eliminated WeightGate for the time being, just to do my due diligence. Also, since there is a fix sized buffer, I assume actually LOWERING the 3rdnumber (the allocation for each non-interactive process) would allow for MORE parallel processes to run (as long as the value is still large enough to support each of the applications that rely on it.) Of course, I assume the heap issue in reality is actually a SECONDARY problem ( a symptom of too many non-interactive tasks being launched and not completing). Since the heap space is finite, there is a hard limit as to how many processes can be in a wait state at the same time. The problem to focus on is not the known, limited heap, but rather the reason why these processes were unable to complete and thus eventually too many processes being active. Indeed. Eliminating WeightGate might impact this because it will represent one less process per message. I just did a search of errors in the SNF logs and didn't find anything unusual. I was unable to pinpoint the time of the problem -- that will require a harder analysis of the data. Indications are that SNFServer didn't see any significant issues during the period covered by the two logs you sent. When client's talked to it they were served (according to the logs). You're showing about 40 msg/minute on average. According to a spot check of log entries SNFServer is finished processing these in an unmeasurable amount of time (0 indicates 15 ms for both setup, read, scan, and response). Most of the logs performance metrics p/ indicate s='0' and t='0' -- setup time in ms, and scan time in ms. On occasion I see some nonzero t values - but nothing unusual (16, 47, 63, etc). You probably don't need a lot of threads active on your system. If you have provided for a high number then you might consider reducing that number... Processing 1 message per second would exceed your average handily and doesn't take a lot of threads. If for some reason you were hit with a large number of messages and put them in work in parallel then that might have exhausted the heap. The new SNF is much more efficient than the old one and so it would have more easily allowed this... Sometimes introducing a more efficient component into a system exposes problems that were hidden by the previous less efficient component -- the less efficient component may have masked the problem by artificially reducing or shaping throughput. When we see this kind of thing we call it a "lens effect" -- the newer component reshapes the dynamics of the system and brings previously unknown problems "into focus". It's possible the heap problem you experienced was caused by a "lens effect" since the new SNF engine is more efficient and would naturally allow for more messages to be handled concurrently in a burst than the previous version would have allowed. A theory -- the previous version would naturally be constrained by I/O contention since it would need to create, scan, modify, and remove job control files. This would naturally couple performance to other I/O intensive operations such as writing new messages to the spool etc. The new version does not have any of this overhead and so would allow for an unconstrained ramp-up of new instances -- that might lead to a higher number of concurrent tasks and cause heap exhaustion--- after heap exhaustion is achieved additional tasks build up in a failed and partially initialized state. This typically continues until the failed tasks are manually removed -- since none of them is ever properly initialized none of the tasks can time out, fail, or shut down on their own. Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
