After following through all of this and looking at the .stat file, I think I see what's going on.
Now that it is running and producing a .stat file, the flow rate is very low. According to the stat data, about 6 msgs / minute. Note the poll and loop times are in the 450 - 550 ms range. SNF with the persistent engine is built for high throughput, but it's also built to play nice. The maximum poll time gets up to 2 seconds or so (sound familiar?) If there are no messages for a while, then everything slows down until the first message goes through. For that first message, the SNF client will probably wait about 2 seconds before looking for it's result because that's what the stat file will tell it to do. Since the next message probably won't come around for a few seconds, that next message will probably wait about 2 seconds also. If you were doing 6 messages a second then all of the times would be much lower and so would the individual delays. When you turn off the persistent instance, each new message causes a client to look and see if there are any other peers acting a servers... Since the messages are far and few between, the client will elect to be a server (momentarily), will find no work but it's own, will process it's own message and leave. -- This is the automatic peer-server mode. It will always work like this unless more than one message is being processed at the same moment. In peer-server mode, since there is nothing else going on and no persistent instance to coordinate the operations, each message will get processed as fast as the rulebase can be loaded and then the program will drop. When the persistent instance is introduced, it sets the pace - and sicne there are no other messages, each client will wait about 2 seconds (or half a second or so with the .stat file contents you show) before it begins looking for it's results. The server instance will also wait a bit before looking for new jobs so that the file system isn't constantly being scanned. Of course, if a burst of messages come through then the pacing will speed up as much as necessary to keep up with the volume. Hope this helps, _M On Tuesday, August 2, 2005, 3:38:52 PM, Dan wrote: DH> No, I followed your instructions exactly (and not for the DH> first time). I didn't add those extra values until today. Prior DH> to adding the AppDirectory value, the service was taking a minute DH> to scan emails; after adding it the scan time went to around 2 DH> seconds. I can't get it any lower than that. Initially mine was DH> set up exactly as you said, with only "Application" containing DH> the path, authcode and persistent. Today after hearing no DH> suggestions from the list, and based on recent list messages DH> mentioning the home directory for the service, I looked at the DH> srvany.exe doco to find out how to give it a home directory. DH> That's when I added AppDirectory. I also saw and added DH> AppParameters at the same time and added those as well, though DH> they seem not to be needed. DH> DH> Prior to adding the AppDirectory value, I never got any DH> .stat file or any .SVR file in my sniffer dir. After adding that DH> value and starting the service those files appeared. DH> DH> DH> From: [EMAIL PROTECTED] DH> [mailto:[EMAIL PROTECTED] On Behalf Of Matt DH> Sent: Tuesday, August 02, 2005 3:24 PM DH> To: sniffer@SortMonster.com DH> Subject: Re: [sniffer] Sniffer taking a long time? DH> Dan, DH> There is no AppDirectory value on my server either. The DH> Parameters key has only one value under it besides Default DH> which is "Application", and it contains exactly what I provided DH> below. Could it be that you tried to hard to get everything DH> right by tweaking these additional keys? DH> Something else. Did you make sure that the Sniffer DH> service that you created was started? No doubt it will work if DH> you follow those directions to a T, and there aren't any issues DH> with your server apart from this. DH> Matt DH> Dan Horne wrote: DH> I removed the AppParameters value and put the authcode DH> and persistent back in the Application value where it was before. DH> It didn't make any difference at all in the processing time, DH> still right around 2 seconds. I don't know how your setup is DH> working without at least the AppDirectory value, because mine DH> didn't start working until I put that in, but if it is, I DH> can't argue. My server load isn't anywhere near yours, so I DH> don't see what the problem could be with mine. Oh well, unless DH> Pete responds with a suggestion, I guess I'll just keep using the DH> non-service version. DH> DH> Thanks anyway. DH> From:[EMAIL PROTECTED] DH> [mailto:[EMAIL PROTECTED] On Behalf Of Matt DH> Sent: Tuesday, August 02, 2005 2:37 PM DH> To:sniffer@SortMonster.com DH> Subject: Re: [sniffer] Sniffer taking a long time? DH> Dan, DH> I seem to recall trying to use the AppParameters key DH> and having difficulty with it. I think that you might want DH> to try removing that key and putting everything in the DH> Parameters key, or at least that works for me. If you change DH> HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters DH> in RegEdit to the following it might fix the issue that you DH> are having: DH> C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe ***AUTH-CODE*** persistent DH> You should of course adjust the path and service name as well. DH> The directions that I provided are working perfectly DH> on my server so far as I can tell. I'm running dual 3.2 DH> Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in RAID 5. DH> The following three debug log entries shows between 300 ms and 550 DH> ms per message: DH> 08/02/2005 14:19:47.113 QB93D976201222A43 [2616] DH> SNIFFER-IP: External program started: DH> C:\IMail\Declude\Sniffer\executable.exe auth-code DH> F:\\DB93D976201222A43.SMD DH> 08/02/2005 14:19:47.676 QB93D976201222A43 [2616] DH> SNIFFER-IP: External program reports exit code of 61 DH> ----------------- DH> 08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196] DH> SNIFFER-IP: External program started: DH> C:\IMail\Declude\Sniffer\executable.exe auth-code DH> F:\\DB9418A4800EC2A49.SMD DH> 08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196] DH> SNIFFER-IP: External program reports exit code of 51 DH> ----------------- DH> 08/02/2005 14:19:49.879 QB943711501382A4D [6388] DH> SNIFFER-IP: External program started: DH> C:\IMail\Declude\Sniffer\executable.exe auth-code DH> F:\\DB943711501382A4D.SMD DH> 08/02/2005 14:19:50.176 QB943711501382A4D [6388] DH> SNIFFER-IP: External program reports exit code of 59 DH> My stat file shows the following: DH> TicToc: 1122992104 DH> Loop: 154 DH> Poll: 0 DH> Jobs: 118392 DH> Secs: 155137 DH> Msg/Min: 45.7887 DH> Current-Load: 24.4275 DH> Average-Load: 23.8719 DH> I'm not sure why people use FireDaemon for this. My DH> experience with SRVANY.exe has been absolutely flawless DH> since I integrated this, and it has worked on both Win2k and DH> Windows 2003. DH> Matt DH> Dan Horne wrote: DH> OK, I have managed to get SOMETHING working, but it still seems too slow DH> and something is still not right. I originally set up the persistent DH> sniffer using the instructions from this DH> post:http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.htmlThis DH> uses SRVANY.exe. I conjectured that possibly the service needed a DH> home directory, so I added an AppDirectory value to the sniffer DH> service's "Parameters" key in the registry. This value is set to the DH> directory sniffer resides in. I also (based on my reading of the DH> srvany.exe documentation) added another value to the same key called DH> AppParameters. This is set to my auth code followed by a space, DH> followed by the word persistent. DH> Now when I start the service, the time spent processing a single message DH> goes down to something around 2 seconds, but is still far longer than DH> the non-service version. I also still had no .stat file in my sniffer DH> directory. I did get a *.SVR file, which I never got before. DH> So then I'm thinking, let's just make sure that I have the latest DH> version of sniffer. I downloaded that, did the necessary renaming of DH> the files and then started the service. NOW there is a DH> *.persistent.stat file. However, the scan time is still at around 2 DH> seconds. DH> Average Scan times (based on average scan times of 5 emails each): DH> Without sniffer service running: .033 seconds DH> With sniffer service running: 2.244 seconds DH> The *.persistent.stat file has the following contents: DH> TicToc: 1122990610 DH> Loop: 512 DH> Poll: 445 DH> Jobs: 34 DH> Secs: 303 DH> Msg/Min: 6.73267 DH> Current-Load: 8.69565 DH> Average-Load: 10.6371 DH> Any suggestions? DH> Thanks, DH> Dan Horne DH> This E-Mail came from the Message Sniffer mailing list. For DH> information and (un)subscription instructions go to DH> http://www.sortmonster.com/MessageSniffer/Help/Help.html DH> -- DH> ===================================================== DH> MailPure custom filters for Declude JunkMail DH> Pro.http://www.mailpure.com/software/===================================================== This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
