RE: Re[2]: [sniffer] Sniffer taking a long time?
Thanks, I will do that. > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Colbeck, Andrew > Sent: Wednesday, August 03, 2005 3:17 AM > To: sniffer@SortMonster.com > Subject: RE: Re[2]: [sniffer] Sniffer taking a long time? > > > So basically, what you are saying is that my volume is > really too low > > to take advantage of the persistent sniffer (and such may actually > > decrease my performance), and I should stick with the non-service > > version. Is that right? That is about what I thought (without the > > details of how sniffer works, I just wanted to be sure). > > Well, Dan, for the inevitable rush of traffic, I'd stick with > the persistent sniffer implementation now that you have it working. > > If the 2 second wait time galls you, then use your **.cfg > file and specify the > > MaxPollTime: 500 > > value at 500 ms or whatever you'd like your maximum wait time > to be instead of 2 seconds (2000 ms). > > Andrew 8) > > > > > This E-Mail came from the Message Sniffer mailing list. For > information and (un)subscription instructions go to > http://www.sortmonster.com/MessageSniffer/Help/Help.html > This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Sniffer taking a long time?
> So basically, what you are saying is that my volume is really > too low to take advantage of the persistent sniffer (and such > may actually decrease my performance), and I should stick > with the non-service version. Is that right? That is about > what I thought (without the details of how sniffer works, I > just wanted to be sure). Well, Dan, for the inevitable rush of traffic, I'd stick with the persistent sniffer implementation now that you have it working. If the 2 second wait time galls you, then use your **.cfg file and specify the MaxPollTime: 500 value at 500 ms or whatever you'd like your maximum wait time to be instead of 2 seconds (2000 ms). Andrew 8) This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Sniffer taking a long time?
Dan, Think about the fact that the entire rulebase and the executable don't have to be read except on occasion when running as a service, and memory doesn't need to be allocated to the application each time it is run. This should also help with the mystery heap issues that can occur on an overloaded IMail/Declude server under very heavy load. If you ever get bursts of traffic, this can come in handy. Matt Dan Horne wrote: So basically, what you are saying is that my volume is really too low to take advantage of the persistent sniffer (and such may actually decrease my performance), and I should stick with the non-service version. Is that right? That is about what I thought (without the details of how sniffer works, I just wanted to be sure). Thanks, Pete. Dan Horne -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Pete McNeil Sent: Tuesday, August 02, 2005 4:09 PM To: Dan Horne Subject: Re[2]: [sniffer] Sniffer taking a long time? After following through all of this and looking at the .stat file, I think I see what's going on. Now that it is running and producing a .stat file, the flow rate is very low. According to the stat data, about 6 msgs / minute. Note the poll and loop times are in the 450 - 550 ms range. SNF with the persistent engine is built for high throughput, but it's also built to play nice. The maximum poll time gets up to 2 seconds or so (sound familiar?) If there are no messages for a while, then everything slows down until the first message goes through. For that first message, the SNF client will probably wait about 2 seconds before looking for it's result because that's what the stat file will tell it to do. Since the next message probably won't come around for a few seconds, that next message will probably wait about 2 seconds also. If you were doing 6 messages a second then all of the times would be much lower and so would the individual delays. When you turn off the persistent instance, each new message causes a client to look and see if there are any other peers acting a servers... Since the messages are far and few between, the client will elect to be a server (momentarily), will find no work but it's own, will process it's own message and leave. -- This is the automatic peer-server mode. It will always work like this unless more than one message is being processed at the same moment. In peer-server mode, since there is nothing else going on and no persistent instance to coordinate the operations, each message will get processed as fast as the rulebase can be loaded and then the program will drop. When the persistent instance is introduced, it sets the pace - and sicne there are no other messages, each client will wait about 2 seconds (or half a second or so with the .stat file contents you show) before it begins looking for it's results. The server instance will also wait a bit before looking for new jobs so that the file system isn't constantly being scanned. Of course, if a burst of messages come through then the pacing will speed up as much as necessary to keep up with the volume. Hope this helps, _M On Tuesday, August 2, 2005, 3:38:52 PM, Dan wrote: DH> No, I followed your instructions exactly (and not for the first DH> time). I didn't add those extra values until today. Prior to DH> adding the AppDirectory value, the service was taking a minute to DH> scan emails; after adding it the scan time went to around 2 DH> seconds. I can't get it any lower than that. Initially mine was DH> set up exactly as you said, with only "Application" containing the DH> path, authcode and persistent. Today after hearing no suggestions DH> from the list, and based on recent list messages mentioning the home DH> directory for the service, I looked at the srvany.exe doco to find DH> out how to give it a home directory. DH> That's when I added AppDirectory. I also saw and added DH> AppParameters at the same time and added those as well, though they DH> seem not to be needed. DH> DH> Prior to adding the AppDirectory value, I never got any .stat file DH> or any .SVR file in my sniffer dir. After adding that value and DH> starting the service those files appeared. DH> DH> DH> From: [EMAIL PROTECTED] DH> [mailto:[EMAIL PROTECTED]] On Behalf Of Matt DH> Sent: Tuesday, August 02, 2005 3:24 PM DH> To: sniffer@SortMonster.com DH> Subject: Re: [sniffer] Sniffer taking a long time? DH> Dan, DH> There is no AppDirectory value on my servereither. The DH> Parameters key has only one value under it besides Default DH> which is "Application", and it contains exactly what I provided DH> below. Could it be that you tried to hard to get everything DH> right by
Re: [sniffer] Sniffer taking a long time?
Dan,
There is no AppDirectory value on my server either. The Parameters key
has only one value under it besides Default which is "Application", and
it contains exactly what I provided below. Could it be that you tried
to hard to get everything right by tweaking these additional keys?
Something else. Did you make sure that the Sniffer service that you
created was started? No doubt it will work if you follow those
directions to a T, and there aren't any issues with your server apart
from this.
Matt
Dan Horne wrote:
I removed the AppParameters
value and put the authcode and persistent back in the Application value
where it was before. It didn't make any difference at all in the
processing time, still right around 2 seconds. I don't know how your
setup is working without at least the AppDirectory value, because mine
didn't start working until I put that in, but if it is, I can't argue.
My server load isn't anywhere near yours, so I don't see what the
problem could be with mine. Oh well, unless Pete responds with a
suggestion, I guess I'll just keep using the non-service version.
Thanks anyway.
From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On
Behalf Of Matt
Sent: Tuesday, August 02, 2005 2:37 PM
To: sniffer@SortMonster.com
Subject: Re: [sniffer] Sniffer taking a long time?
Dan,
I seem to recall trying to use the AppParameters key and having
difficulty with it. I think that you might want to try removing that
key and putting everything in the Parameters key, or at least that
works for me. If you change
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in
RegEdit to the following it might fix the issue that you are having:
C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe
***AUTH-CODE*** persistent
You should of course adjust the path and service name as well.
The directions that I provided are working perfectly on my server so
far as I can tell. I'm running dual 3.2 Ghz 1 MB cache Xeons with 5 x
15,000 RPM drives in RAID 5. The following three debug log entries
shows between 300 ms and 550 ms per message:
08/02/2005 14:19:47.113 QB93D976201222A43 [2616]
SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe
auth-code F:\\DB93D976201222A43.SMD
08/02/2005 14:19:47.676 QB93D976201222A43 [2616]
SNIFFER-IP: External program reports exit code of 61
-
08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196]
SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe
auth-code F:\\DB9418A4800EC2A49.SMD
08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196]
SNIFFER-IP: External program reports exit code of 51
-
08/02/2005 14:19:49.879 QB943711501382A4D [6388]
SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe
auth-code F:\\DB943711501382A4D.SMD
08/02/2005 14:19:50.176 QB943711501382A4D [6388]
SNIFFER-IP: External program reports exit code of 59
My stat file shows the following:
TicToc: 1122992104
Loop: 154
Poll: 0
Jobs: 118392
Secs: 155137
Msg/Min: 45.7887
Current-Load: 24.4275
Average-Load: 23.8719
I'm not sure why people use FireDaemon for this. My experience with
SRVANY.exe has been absolutely flawless since I integrated this, and it
has worked on both Win2k and Windows 2003.
Matt
Dan Horne wrote:
OK, I have managed to get SOMETHING working, but it still seems too slow
and something is still not right. I originally set up the persistent
sniffer using the instructions from this post:
http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.html
This uses SRVANY.exe. I conjectured that possibly the service needed a
home directory, so I added an AppDirectory value to the sniffer
service's "Parameters" key in the registry. This value is set to the
directory sniffer resides in. I also (based on my reading of the
srvany.exe documentation) added another value to the same key called
AppParameters. This is set to my auth code followed by a space,
followed by the word persistent.
Now when I start the service, the time spent processing a single message
goes down to something around 2 seconds, but is still far longer than
the non-service version. I also still had no .stat file in my sniffer
directory. I did get a *.SVR file, which I never got before.
So then I'm thinking, let's just make sure that I have the latest
version of sniffer. I downloaded that, did the necessary renaming of
the files and then started the service. NOW there is a
*.persistent.stat file. However, the scan time is still at around 2
seconds.
Average Scan times (based on average scan times of 5 emails each):
Without sniffer service running: .033 seconds
With sniffer service running: 2.244 seconds
The *.persistent.stat file has
Re[4]: [sniffer] Sniffer taking a long time?
Well, it's not going to hurt your performance at all (a 2 second delay on each email is not going to be noticed in most cases - email is not IM after all). That said, the persistent mode is not necessary either though It will help if you get a burst of high activity. _M On Tuesday, August 2, 2005, 4:24:11 PM, Dan wrote: DH> So basically, what you are saying is that my volume is really DH> too low to take advantage of the persistent sniffer (and such may DH> actually decrease my performance), and I should stick with the DH> non-service version. Is that right? That is about what I thought DH> (without the details of how sniffer works, I just wanted to be DH> sure). DH> Thanks, Pete. DH> Dan Horne >> -Original Message- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil >> Sent: Tuesday, August 02, 2005 4:09 PM >> To: Dan Horne >> Subject: Re[2]: [sniffer] Sniffer taking a long time? >> >> After following through all of this and looking at the .stat >> file, I think I see what's going on. >> >> Now that it is running and producing a .stat file, the flow >> rate is very low. According to the stat data, about 6 msgs / minute. >> >> Note the poll and loop times are in the 450 - 550 ms range. >> >> SNF with the persistent engine is built for high throughput, >> but it's also built to play nice. >> >> The maximum poll time gets up to 2 seconds or so (sound familiar?) >> >> If there are no messages for a while, then everything slows >> down until the first message goes through. For that first >> message, the SNF client will probably wait about 2 seconds >> before looking for it's result because that's what the stat >> file will tell it to do. >> >> Since the next message probably won't come around for a few >> seconds, that next message will probably wait about 2 seconds also. >> >> If you were doing 6 messages a second then all of the times >> would be much lower and so would the individual delays. >> >> When you turn off the persistent instance, each new message >> causes a client to look and see if there are any other peers >> acting a servers... Since the messages are far and few >> between, the client will elect to be a server (momentarily), >> will find no work but it's own, will process it's own message >> and leave. -- This is the automatic peer-server mode. It will >> always work like this unless more than one message is being >> processed at the same moment. >> >> In peer-server mode, since there is nothing else going on and >> no persistent instance to coordinate the operations, each >> message will get processed as fast as the rulebase can be >> loaded and then the program will drop. >> >> When the persistent instance is introduced, it sets the pace >> - and sicne there are no other messages, each client will >> wait about 2 seconds (or half a second or so with the .stat >> file contents you show) before it begins looking for it's results. >> >> The server instance will also wait a bit before looking for >> new jobs so that the file system isn't constantly being scanned. >> >> Of course, if a burst of messages come through then the >> pacing will speed up as much as necessary to keep up with the volume. >> >> Hope this helps, >> >> _M >> >> On Tuesday, August 2, 2005, 3:38:52 PM, Dan wrote: >> >> DH> No, I followed your instructions exactly (and not for the first >> DH> time). I didn't add those extra values until today. Prior to >> DH> adding the AppDirectory value, the service was taking a minute to >> DH> scan emails; after adding it the scan time went to around 2 >> DH> seconds. I can't get it any lower than that. Initially >> mine was >> DH> set up exactly as you said, with only "Application" >> containing the >> DH> path, authcode and persistent. Today after hearing no >> suggestions >> DH> from the list, and based on recent list messages >> mentioning the home >> DH> directory for the service, I looked at the srvany.exe >> doco to find >> DH> out how to give it a home directory. >> DH> That's when I added AppDirectory. I also saw and added >> DH> AppParameters at the same time and added those as well, >> though they >> DH> seem not to be needed. >> DH> >> DH> Prior to adding the AppDirectory value, I never got any >> .stat file >&g
RE: [sniffer] Sniffer taking a long time?
Nah, the .cfg file is default and srvany.exe was installed
with the resource kit.
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
MattSent: Tuesday, August 02, 2005 3:58 PMTo:
sniffer@SortMonster.comSubject: Re: [sniffer] Sniffer taking a long
time?
Dan,Just two thoughts at this moment. One would be
permissions. Maybe setting up SRVANY by installing the Resource Kit
would be the way to go (if that isn't what you did). I would imagine
that if the service starts, all should be fine. The other would be to
check your Sniffer's CFG file to make sure that nothing funky is in
there.MattDan Horne wrote:
No, I followed your instructions exactly (and not for
the first time). I didn't add those extra values until today.
Prior to adding the AppDirectory value, the service was taking a minute to
scan emails; after adding it the scan time went to around 2 seconds. I
can't get it any lower than that. Initially mine was set up exactly as
you said, with only "Application" containing the path, authcode and
persistent. Today after hearing no suggestions from the list, and
based on recent list messages mentioning the home directory for the service,
I looked at the srvany.exe doco to find out how to give it a home
directory. That's when I added AppDirectory. I also saw and
added AppParameters at the same time and added those as well, though they
seem not to be needed.
Prior to adding the AppDirectory value, I never got any
.stat file or any .SVR file in my sniffer dir. After adding that value
and starting the service those files appeared.
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]
On Behalf Of MattSent: Tuesday, August 02, 2005 3:24
PMTo: sniffer@SortMonster.comSubject:
Re: [sniffer] Sniffer taking a long time?
Dan,There
is no AppDirectory value on my server either. The Parameters key has
only one value under it besides Default which is "Application", and it
contains exactly what I provided below. Could it be that you tried
to hard to get everything right by tweaking these additional
keys?Something else. Did you make sure that the Sniffer
service that you created was started? No doubt it will work if you
follow those directions to a T, and there aren't any issues with your
server apart from this.MattDan Horne wrote:
I removed the AppParameters value and put the
authcode and persistent back in the Application value where it was
before. It didn't make any difference at all in the processing
time, still right around 2 seconds. I don't know how your setup is
working without at least the AppDirectory value, because mine didn't
start working until I put that in, but if it is, I can't argue. My
server load isn't anywhere near yours, so I don't see what the problem
could be with mine. Oh well, unless Pete responds with a
suggestion, I guess I'll just keep using the non-service
version.
Thanks anyway.
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]
On Behalf Of MattSent: Tuesday, August 02, 2005 2:37
PMTo: sniffer@SortMonster.comSubject:
Re: [sniffer] Sniffer taking a long
time?Dan,I seem to recall trying to use
the AppParameters key and having difficulty with it. I think
that you might want to try removing that key and putting everything in
the Parameters key, or at least that works for me. If you change
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in
RegEdit to the following it might fix the issue that you are
having:
C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe
***AUTH-CODE***
persistentYou should of course adjust the
path and service name as well.The directions that I provided
are working perfectly on my server so far as I can tell. I'm
running dual 3.2 Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in
RAID 5. The following three debug log entries shows between 300
ms and 550 ms per message:
08/02/2005 14:19:47.113 QB93D976201222A43 [2616]
SNIFFER-IP: External program started:
C:\IMail\Declude\Sniffer\executable.exe
auth-code
F:\\DB93D976201222A43.SMD08/02/2005 14:19:47.676
QB93D976201222A43 [2616] SNIFFER-IP: External program reports exit
code of
61-08/02/2005
14:19:47.488 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program
RE: Re[2]: [sniffer] Sniffer taking a long time?
So basically, what you are saying is that my volume is really too low to take advantage of the persistent sniffer (and such may actually decrease my performance), and I should stick with the non-service version. Is that right? That is about what I thought (without the details of how sniffer works, I just wanted to be sure). Thanks, Pete. Dan Horne > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil > Sent: Tuesday, August 02, 2005 4:09 PM > To: Dan Horne > Subject: Re[2]: [sniffer] Sniffer taking a long time? > > After following through all of this and looking at the .stat > file, I think I see what's going on. > > Now that it is running and producing a .stat file, the flow > rate is very low. According to the stat data, about 6 msgs / minute. > > Note the poll and loop times are in the 450 - 550 ms range. > > SNF with the persistent engine is built for high throughput, > but it's also built to play nice. > > The maximum poll time gets up to 2 seconds or so (sound familiar?) > > If there are no messages for a while, then everything slows > down until the first message goes through. For that first > message, the SNF client will probably wait about 2 seconds > before looking for it's result because that's what the stat > file will tell it to do. > > Since the next message probably won't come around for a few > seconds, that next message will probably wait about 2 seconds also. > > If you were doing 6 messages a second then all of the times > would be much lower and so would the individual delays. > > When you turn off the persistent instance, each new message > causes a client to look and see if there are any other peers > acting a servers... Since the messages are far and few > between, the client will elect to be a server (momentarily), > will find no work but it's own, will process it's own message > and leave. -- This is the automatic peer-server mode. It will > always work like this unless more than one message is being > processed at the same moment. > > In peer-server mode, since there is nothing else going on and > no persistent instance to coordinate the operations, each > message will get processed as fast as the rulebase can be > loaded and then the program will drop. > > When the persistent instance is introduced, it sets the pace > - and sicne there are no other messages, each client will > wait about 2 seconds (or half a second or so with the .stat > file contents you show) before it begins looking for it's results. > > The server instance will also wait a bit before looking for > new jobs so that the file system isn't constantly being scanned. > > Of course, if a burst of messages come through then the > pacing will speed up as much as necessary to keep up with the volume. > > Hope this helps, > > _M > > On Tuesday, August 2, 2005, 3:38:52 PM, Dan wrote: > > DH> No, I followed your instructions exactly (and not for the first > DH> time). I didn't add those extra values until today. Prior to > DH> adding the AppDirectory value, the service was taking a minute to > DH> scan emails; after adding it the scan time went to around 2 > DH> seconds. I can't get it any lower than that. Initially > mine was > DH> set up exactly as you said, with only "Application" > containing the > DH> path, authcode and persistent. Today after hearing no > suggestions > DH> from the list, and based on recent list messages > mentioning the home > DH> directory for the service, I looked at the srvany.exe > doco to find > DH> out how to give it a home directory. > DH> That's when I added AppDirectory. I also saw and added > DH> AppParameters at the same time and added those as well, > though they > DH> seem not to be needed. > DH> > DH> Prior to adding the AppDirectory value, I never got any > .stat file > DH> or any .SVR file in my sniffer dir. After adding that value and > DH> starting the service those files appeared. > DH> > DH> > > > DH> From: [EMAIL PROTECTED] > DH> [mailto:[EMAIL PROTECTED] On Behalf Of Matt > DH> Sent: Tuesday, August 02, 2005 3:24 PM > DH> To: sniffer@SortMonster.com > DH> Subject: Re: [sniffer] Sniffer taking a long time? > > > > > DH> Dan, > > DH> There is no AppDirectory value on my servereither. The > DH> Parameters key has only one value under it besides Default > DH> which is "Application", and it contains exactly what I provided > DH> below.
Re[2]: [sniffer] Sniffer taking a long time?
After following through all of this and looking at the .stat file, I think I see what's going on. Now that it is running and producing a .stat file, the flow rate is very low. According to the stat data, about 6 msgs / minute. Note the poll and loop times are in the 450 - 550 ms range. SNF with the persistent engine is built for high throughput, but it's also built to play nice. The maximum poll time gets up to 2 seconds or so (sound familiar?) If there are no messages for a while, then everything slows down until the first message goes through. For that first message, the SNF client will probably wait about 2 seconds before looking for it's result because that's what the stat file will tell it to do. Since the next message probably won't come around for a few seconds, that next message will probably wait about 2 seconds also. If you were doing 6 messages a second then all of the times would be much lower and so would the individual delays. When you turn off the persistent instance, each new message causes a client to look and see if there are any other peers acting a servers... Since the messages are far and few between, the client will elect to be a server (momentarily), will find no work but it's own, will process it's own message and leave. -- This is the automatic peer-server mode. It will always work like this unless more than one message is being processed at the same moment. In peer-server mode, since there is nothing else going on and no persistent instance to coordinate the operations, each message will get processed as fast as the rulebase can be loaded and then the program will drop. When the persistent instance is introduced, it sets the pace - and sicne there are no other messages, each client will wait about 2 seconds (or half a second or so with the .stat file contents you show) before it begins looking for it's results. The server instance will also wait a bit before looking for new jobs so that the file system isn't constantly being scanned. Of course, if a burst of messages come through then the pacing will speed up as much as necessary to keep up with the volume. Hope this helps, _M On Tuesday, August 2, 2005, 3:38:52 PM, Dan wrote: DH> No, I followed your instructions exactly (and not for the DH> first time). I didn't add those extra values until today. Prior DH> to adding the AppDirectory value, the service was taking a minute DH> to scan emails; after adding it the scan time went to around 2 DH> seconds. I can't get it any lower than that. Initially mine was DH> set up exactly as you said, with only "Application" containing DH> the path, authcode and persistent. Today after hearing no DH> suggestions from the list, and based on recent list messages DH> mentioning the home directory for the service, I looked at the DH> srvany.exe doco to find out how to give it a home directory. DH> That's when I added AppDirectory. I also saw and added DH> AppParameters at the same time and added those as well, though DH> they seem not to be needed. DH> DH> Prior to adding the AppDirectory value, I never got any DH> .stat file or any .SVR file in my sniffer dir. After adding that DH> value and starting the service those files appeared. DH> DH> DH> From: [EMAIL PROTECTED] DH> [mailto:[EMAIL PROTECTED] On Behalf Of Matt DH> Sent: Tuesday, August 02, 2005 3:24 PM DH> To: sniffer@SortMonster.com DH> Subject: Re: [sniffer] Sniffer taking a long time? DH> Dan, DH> There is no AppDirectory value on my servereither. The DH> Parameters key has only one value under it besides Default DH> which is "Application", and it contains exactly what I provided DH> below. Could it be that you tried to hard to get everything DH> right by tweaking theseadditional keys? DH> Something else. Did you make sure that theSniffer DH> service that you created was started? No doubt it will work if DH> you follow those directions to a T, and there aren't any issues DH> with yourserver apart from this. DH> Matt DH> Dan Horne wrote: DH> I removed the AppParameters value and put the authcode DH> and persistent back in the Application value where it was before. DH> It didn't make any difference at all in the processing time, DH> still right around 2 seconds. I don't know how your setup is DH> working without at least the AppDirectory value, because mine DH> didn't start working until I put that in, but if it is, I DH> can't argue. My server load isn't anywhere near yours, so I DH> don't see what the problem could be with mine. Oh well, unless DH> Pete responds with a suggestion, I guess I'll just keep using the DH> non-service version. DH> DH&g
Re: [sniffer] Sniffer taking a long time?
You are correct. My bad. Matt Nick Hayer wrote: Without regard to content I believe the edits would be made in CurrentControlSet - not in ControlSetxxx - the later are the backups. -Nick Matt wrote: Dan, I seem to recall trying to use the AppParameters key and having difficulty with it. I think that you might want to try removing that key and putting everything in the Parameters key, or at least that works for me. If you change HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in RegEdit to the following it might fix the issue that you are having: C:\IMail\Declude\Sniffer\ ***RULEBASE-NAME*** .exe ***AUTH-CODE*** persistent You should of course adjust the path and service name as well. The directions that I provided are working perfectly on my server so far as I can tell. I'm running dual 3.2 Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in RAID 5. The following three debug log entries shows between 300 ms and 550 ms per message: 08/02/2005 14:19:47.113 QB93D976201222A43 [2616] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\ executable .exe auth-code F:\\DB93D976201222A43.SMD 08/02/2005 14:19:47.676 QB93D976201222A43 [2616] SNIFFER-IP: External program reports exit code of 61 - 08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\ executable .exe auth-code F:\\DB9418A4800EC2A49.SMD 08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program reports exit code of 51 - 08/02/2005 14:19:49.879 QB943711501382A4D [6388] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\ executable .exe auth-code F:\\DB943711501382A4D.SMD 08/02/2005 14:19:50.176 QB943711501382A4D [6388] SNIFFER-IP: External program reports exit code of 59 My stat file shows the following: TicToc: 1122992104 Loop: 154 Poll: 0 Jobs: 118392 Secs: 155137 Msg/Min: 45.7887 Current-Load: 24.4275 Average-Load: 23.8719 I'm not sure why people use FireDaemon for this. My experience with SRVANY.exe has been absolutely flawless since I integrated this, and it has worked on both Win2k and Windows 2003. Matt Dan Horne wrote: OK, I have managed to get SOMETHING working, but it still seems too slow and something is still not right. I originally set up the persistent sniffer using the instructions from this post: http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.html This uses SRVANY.exe. I conjectured that possibly the service needed a home directory, so I added an AppDirectory value to the sniffer service's "Parameters" key in the registry. This value is set to the directory sniffer resides in. I also (based on my reading of the srvany.exe documentation) added another value to the same key called AppParameters. This is set to my auth code followed by a space, followed by the word persistent. Now when I start the service, the time spent processing a single message goes down to something around 2 seconds, but is still far longer than the non-service version. I also still had no .stat file in my sniffer directory. I did get a *.SVR file, which I never got before. So then I'm thinking, let's just make sure that I have the latest version of sniffer. I downloaded that, did the necessary renaming of the files and then started the service. NOW there is a *.persistent.stat file. However, the scan time is still at around 2 seconds. Average Scan times (based on average scan times of 5 emails each): Without sniffer service running: .033 seconds With sniffer service running: 2.244 seconds The *.persistent.stat file has the following contents: TicToc: 1122990610 Loop: 512 Poll: 445 Jobs: 34 Secs: 303 Msg/Min: 6.73267 Current-Load: 8.69565 Average-Load: 10.6371 Any suggestions? Thanks, Dan Horne This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re: [sniffer] Sniffer taking a long time?
Dan,
Just two thoughts at this moment. One would be permissions. Maybe
setting up SRVANY by installing the Resource Kit would be the way to go
(if that isn't what you did). I would imagine that if the service
starts, all should be fine. The other would be to check your Sniffer's
CFG file to make sure that nothing funky is in there.
Matt
Dan Horne wrote:
No, I followed your instructions
exactly (and not for the first time). I didn't add those extra values
until today. Prior to adding the AppDirectory value, the service was
taking a minute to scan emails; after adding it the scan time went to
around 2 seconds. I can't get it any lower than that. Initially mine
was set up exactly as you said, with only "Application" containing the
path, authcode and persistent. Today after hearing no suggestions from
the list, and based on recent list messages mentioning the home
directory for the service, I looked at the srvany.exe doco to find out
how to give it a home directory. That's when I added AppDirectory. I
also saw and added AppParameters at the same time and added those as
well, though they seem not to be needed.
Prior to adding the AppDirectory
value, I never got any .stat file or any .SVR file in my sniffer dir.
After adding that value and starting the service those files appeared.
From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On
Behalf Of Matt
Sent: Tuesday, August 02, 2005 3:24 PM
To: sniffer@SortMonster.com
Subject: Re: [sniffer] Sniffer taking a long time?
Dan,
There is no AppDirectory value on my server either. The Parameters key
has only one value under it besides Default which is "Application", and
it contains exactly what I provided below. Could it be that you tried
to hard to get everything right by tweaking these additional keys?
Something else. Did you make sure that the Sniffer service that you
created was started? No doubt it will work if you follow those
directions to a T, and there aren't any issues with your server apart
from this.
Matt
Dan Horne wrote:
I removed the AppParameters
value and put the authcode and persistent back in the Application value
where it was before. It didn't make any difference at all in the
processing time, still right around 2 seconds. I don't know how your
setup is working without at least the AppDirectory value, because mine
didn't start working until I put that in, but if it is, I can't argue.
My server load isn't anywhere near yours, so I don't see what the
problem could be with mine. Oh well, unless Pete responds with a
suggestion, I guess I'll just keep using the non-service version.
Thanks anyway.
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]
On Behalf Of Matt
Sent: Tuesday, August 02, 2005 2:37 PM
To: sniffer@SortMonster.com
Subject: Re: [sniffer] Sniffer taking a long time?
Dan,
I seem to recall trying to use the AppParameters key and having
difficulty with it. I think that you might want to try removing that
key and putting everything in the Parameters key, or at least that
works for me. If you change
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in
RegEdit to the following it might fix the issue that you are having:
C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe
***AUTH-CODE*** persistent
You should of course adjust the path and service name as well.
The directions that I provided are working perfectly on my server so
far as I can tell. I'm running dual 3.2 Ghz 1 MB cache Xeons with 5 x
15,000 RPM drives in RAID 5. The following three debug log entries
shows between 300 ms and 550 ms per message:
08/02/2005 14:19:47.113 QB93D976201222A43
[2616] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe
auth-code
F:\\DB93D976201222A43.SMD
08/02/2005 14:19:47.676 QB93D976201222A43 [2616]
SNIFFER-IP: External program reports exit code of 61
-
08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196]
SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe
auth-code
F:\\DB9418A4800EC2A49.SMD
08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196]
SNIFFER-IP: External program reports exit code of 51
-
08/02/2005 14:19:49.879 QB943711501382A4D [6388]
SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe
auth-code
F:\\DB943711501382A4D.SMD
08/02/2005 14:19:50.176 QB943711501382A4D [6388]
SNIFFER-IP: External program reports exit code of 59
My stat file shows the following:
TicToc: 1122992104
Loop: 154
Poll: 0
Jobs: 118392
Secs: 155137
Msg/Min: 45.7887
Current-Load: 24.427
Re: [sniffer] Sniffer taking a long time?
Without regard to content I believe the edits would be made in CurrentControlSet - not in ControlSetxxx - the later are the backups. -Nick Matt wrote: Dan, I seem to recall trying to use the AppParameters key and having difficulty with it. I think that you might want to try removing that key and putting everything in the Parameters key, or at least that works for me. If you change HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in RegEdit to the following it might fix the issue that you are having: C:\IMail\Declude\Sniffer\ ***RULEBASE-NAME*** .exe ***AUTH-CODE*** persistent You should of course adjust the path and service name as well. The directions that I provided are working perfectly on my server so far as I can tell. I'm running dual 3.2 Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in RAID 5. The following three debug log entries shows between 300 ms and 550 ms per message: 08/02/2005 14:19:47.113 QB93D976201222A43 [2616] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\ executable .exe auth-code F:\\DB93D976201222A43.SMD 08/02/2005 14:19:47.676 QB93D976201222A43 [2616] SNIFFER-IP: External program reports exit code of 61 - 08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\ executable .exe auth-code F:\\DB9418A4800EC2A49.SMD 08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program reports exit code of 51 - 08/02/2005 14:19:49.879 QB943711501382A4D [6388] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\ executable .exe auth-code F:\\DB943711501382A4D.SMD 08/02/2005 14:19:50.176 QB943711501382A4D [6388] SNIFFER-IP: External program reports exit code of 59 My stat file shows the following: TicToc: 1122992104 Loop: 154 Poll: 0 Jobs: 118392 Secs: 155137 Msg/Min: 45.7887 Current-Load: 24.4275 Average-Load: 23.8719 I'm not sure why people use FireDaemon for this. My experience with SRVANY.exe has been absolutely flawless since I integrated this, and it has worked on both Win2k and Windows 2003. Matt Dan Horne wrote: OK, I have managed to get SOMETHING working, but it still seems too slow and something is still not right. I originally set up the persistent sniffer using the instructions from this post: http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.html This uses SRVANY.exe. I conjectured that possibly the service needed a home directory, so I added an AppDirectory value to the sniffer service's "Parameters" key in the registry. This value is set to the directory sniffer resides in. I also (based on my reading of the srvany.exe documentation) added another value to the same key called AppParameters. This is set to my auth code followed by a space, followed by the word persistent. Now when I start the service, the time spent processing a single message goes down to something around 2 seconds, but is still far longer than the non-service version. I also still had no .stat file in my sniffer directory. I did get a *.SVR file, which I never got before. So then I'm thinking, let's just make sure that I have the latest version of sniffer. I downloaded that, did the necessary renaming of the files and then started the service. NOW there is a *.persistent.stat file. However, the scan time is still at around 2 seconds. Average Scan times (based on average scan times of 5 emails each): Without sniffer service running: .033 seconds With sniffer service running: 2.244 seconds The *.persistent.stat file has the following contents: TicToc: 1122990610 Loop: 512 Poll: 445 Jobs: 34 Secs: 303 Msg/Min: 6.73267 Current-Load: 8.69565 Average-Load: 10.6371 Any suggestions? Thanks, Dan Horne This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
RE: [sniffer] Sniffer taking a long time?
No, I followed your instructions exactly (and not for the first time). I didn't add those extra values until today. Prior to adding the AppDirectory value, the service was taking a minute to scan emails; after adding it the scan time went to around 2 seconds. I can't get it any lower than that. Initially mine was set up exactly as you said, with only "Application" containing the path, authcode and persistent. Today after hearing no suggestions from the list, and based on recent list messages mentioning the home directory for the service, I looked at the srvany.exe doco to find out how to give it a home directory. That's when I added AppDirectory. I also saw and added AppParameters at the same time and added those as well, though they seem not to be needed. Prior to adding the AppDirectory value, I never got any .stat file or any .SVR file in my sniffer dir. After adding that value and starting the service those files appeared. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MattSent: Tuesday, August 02, 2005 3:24 PMTo: sniffer@SortMonster.comSubject: Re: [sniffer] Sniffer taking a long time? Dan,There is no AppDirectory value on my server either. The Parameters key has only one value under it besides Default which is "Application", and it contains exactly what I provided below. Could it be that you tried to hard to get everything right by tweaking these additional keys?Something else. Did you make sure that the Sniffer service that you created was started? No doubt it will work if you follow those directions to a T, and there aren't any issues with your server apart from this.MattDan Horne wrote: I removed the AppParameters value and put the authcode and persistent back in the Application value where it was before. It didn't make any difference at all in the processing time, still right around 2 seconds. I don't know how your setup is working without at least the AppDirectory value, because mine didn't start working until I put that in, but if it is, I can't argue. My server load isn't anywhere near yours, so I don't see what the problem could be with mine. Oh well, unless Pete responds with a suggestion, I guess I'll just keep using the non-service version. Thanks anyway. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of MattSent: Tuesday, August 02, 2005 2:37 PMTo: sniffer@SortMonster.comSubject: Re: [sniffer] Sniffer taking a long time?Dan,I seem to recall trying to use the AppParameters key and having difficulty with it. I think that you might want to try removing that key and putting everything in the Parameters key, or at least that works for me. If you change HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in RegEdit to the following it might fix the issue that you are having: C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe ***AUTH-CODE*** persistentYou should of course adjust the path and service name as well.The directions that I provided are working perfectly on my server so far as I can tell. I'm running dual 3.2 Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in RAID 5. The following three debug log entries shows between 300 ms and 550 ms per message: 08/02/2005 14:19:47.113 QB93D976201222A43 [2616] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB93D976201222A43.SMD08/02/2005 14:19:47.676 QB93D976201222A43 [2616] SNIFFER-IP: External program reports exit code of 61-08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB9418A4800EC2A49.SMD08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program reports exit code of 51-08/02/2005 14:19:49.879 QB943711501382A4D [6388] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB943711501382A4D.SMD08/02/2005 14:19:50.176 QB943711501382A4D [6388] SNIFFER-IP: External program reports exit code of 59My stat file shows the following: TicToc: 1122992104Loop: 154Poll: 0Jobs: 118392Secs: 155137Msg/Min: 45.7887Current-Load: 24.4275 Average-Load: 23.8719 I'm not sure why people use FireDaemon for this. My experience with SRVANY.exe has been absolutely flawless since I integrated this, and it has worked on both Win2k and Windows 2003.MattDan Horne wrote: OK, I have managed to ge
RE: [sniffer] Sniffer taking a long time?
I removed the AppParameters value and put the authcode and persistent back in the Application value where it was before. It didn't make any difference at all in the processing time, still right around 2 seconds. I don't know how your setup is working without at least the AppDirectory value, because mine didn't start working until I put that in, but if it is, I can't argue. My server load isn't anywhere near yours, so I don't see what the problem could be with mine. Oh well, unless Pete responds with a suggestion, I guess I'll just keep using the non-service version. Thanks anyway. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MattSent: Tuesday, August 02, 2005 2:37 PMTo: sniffer@SortMonster.comSubject: Re: [sniffer] Sniffer taking a long time? Dan,I seem to recall trying to use the AppParameters key and having difficulty with it. I think that you might want to try removing that key and putting everything in the Parameters key, or at least that works for me. If you change HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in RegEdit to the following it might fix the issue that you are having: C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe ***AUTH-CODE*** persistentYou should of course adjust the path and service name as well.The directions that I provided are working perfectly on my server so far as I can tell. I'm running dual 3.2 Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in RAID 5. The following three debug log entries shows between 300 ms and 550 ms per message: 08/02/2005 14:19:47.113 QB93D976201222A43 [2616] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB93D976201222A43.SMD08/02/2005 14:19:47.676 QB93D976201222A43 [2616] SNIFFER-IP: External program reports exit code of 61-08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB9418A4800EC2A49.SMD08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program reports exit code of 51-08/02/2005 14:19:49.879 QB943711501382A4D [6388] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB943711501382A4D.SMD08/02/2005 14:19:50.176 QB943711501382A4D [6388] SNIFFER-IP: External program reports exit code of 59My stat file shows the following: TicToc: 1122992104Loop: 154Poll: 0Jobs: 118392Secs: 155137Msg/Min: 45.7887Current-Load: 24.4275 Average-Load: 23.8719 I'm not sure why people use FireDaemon for this. My experience with SRVANY.exe has been absolutely flawless since I integrated this, and it has worked on both Win2k and Windows 2003.MattDan Horne wrote: OK, I have managed to get SOMETHING working, but it still seems too slow and something is still not right. I originally set up the persistent sniffer using the instructions from this post: http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.html This uses SRVANY.exe. I conjectured that possibly the service needed a home directory, so I added an AppDirectory value to the sniffer service's "Parameters" key in the registry. This value is set to the directory sniffer resides in. I also (based on my reading of the srvany.exe documentation) added another value to the same key called AppParameters. This is set to my auth code followed by a space, followed by the word persistent. Now when I start the service, the time spent processing a single message goes down to something around 2 seconds, but is still far longer than the non-service version. I also still had no .stat file in my sniffer directory. I did get a *.SVR file, which I never got before. So then I'm thinking, let's just make sure that I have the latest version of sniffer. I downloaded that, did the necessary renaming of the files and then started the service. NOW there is a *.persistent.stat file. However, the scan time is still at around 2 seconds. Average Scan times (based on average scan times of 5 emails each): Without sniffer service running: .033 seconds With sniffer service running: 2.244 seconds The *.persistent.stat file has the following contents: TicToc: 1122990610 Loop: 512 Poll: 445 Jobs: 34 Secs: 303 Msg/Min: 6.73267 Current-Load: 8.69565 Average-Load: 10.6371 Any suggestions? Thanks, Dan Horne This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re: [sniffer] Sniffer taking a long time?
Dan, I seem to recall trying to use the AppParameters key and having difficulty with it. I think that you might want to try removing that key and putting everything in the Parameters key, or at least that works for me. If you change HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Sniffer\Parameters in RegEdit to the following it might fix the issue that you are having: C:\IMail\Declude\Sniffer\***RULEBASE-NAME***.exe ***AUTH-CODE*** persistent You should of course adjust the path and service name as well. The directions that I provided are working perfectly on my server so far as I can tell. I'm running dual 3.2 Ghz 1 MB cache Xeons with 5 x 15,000 RPM drives in RAID 5. The following three debug log entries shows between 300 ms and 550 ms per message: 08/02/2005 14:19:47.113 QB93D976201222A43 [2616] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB93D976201222A43.SMD 08/02/2005 14:19:47.676 QB93D976201222A43 [2616] SNIFFER-IP: External program reports exit code of 61 - 08/02/2005 14:19:47.488 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB9418A4800EC2A49.SMD 08/02/2005 14:19:47.770 QB9418A4800EC2A49 [6196] SNIFFER-IP: External program reports exit code of 51 - 08/02/2005 14:19:49.879 QB943711501382A4D [6388] SNIFFER-IP: External program started: C:\IMail\Declude\Sniffer\executable.exe auth-code F:\\DB943711501382A4D.SMD 08/02/2005 14:19:50.176 QB943711501382A4D [6388] SNIFFER-IP: External program reports exit code of 59 My stat file shows the following: TicToc: 1122992104 Loop: 154 Poll: 0 Jobs: 118392 Secs: 155137 Msg/Min: 45.7887 Current-Load: 24.4275 Average-Load: 23.8719 I'm not sure why people use FireDaemon for this. My experience with SRVANY.exe has been absolutely flawless since I integrated this, and it has worked on both Win2k and Windows 2003. Matt Dan Horne wrote: OK, I have managed to get SOMETHING working, but it still seems too slow and something is still not right. I originally set up the persistent sniffer using the instructions from this post: http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.html This uses SRVANY.exe. I conjectured that possibly the service needed a home directory, so I added an AppDirectory value to the sniffer service's "Parameters" key in the registry. This value is set to the directory sniffer resides in. I also (based on my reading of the srvany.exe documentation) added another value to the same key called AppParameters. This is set to my auth code followed by a space, followed by the word persistent. Now when I start the service, the time spent processing a single message goes down to something around 2 seconds, but is still far longer than the non-service version. I also still had no .stat file in my sniffer directory. I did get a *.SVR file, which I never got before. So then I'm thinking, let's just make sure that I have the latest version of sniffer. I downloaded that, did the necessary renaming of the files and then started the service. NOW there is a *.persistent.stat file. However, the scan time is still at around 2 seconds. Average Scan times (based on average scan times of 5 emails each): Without sniffer service running: .033 seconds With sniffer service running: 2.244 seconds The *.persistent.stat file has the following contents: TicToc: 1122990610 Loop: 512 Poll: 445 Jobs: 34 Secs: 303 Msg/Min: 6.73267 Current-Load: 8.69565 Average-Load: 10.6371 Any suggestions? Thanks, Dan Horne This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
RE: [sniffer] Sniffer taking a long time?
OK, I have managed to get SOMETHING working, but it still seems too slow and something is still not right. I originally set up the persistent sniffer using the instructions from this post: http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.html This uses SRVANY.exe. I conjectured that possibly the service needed a home directory, so I added an AppDirectory value to the sniffer service's "Parameters" key in the registry. This value is set to the directory sniffer resides in. I also (based on my reading of the srvany.exe documentation) added another value to the same key called AppParameters. This is set to my auth code followed by a space, followed by the word persistent. Now when I start the service, the time spent processing a single message goes down to something around 2 seconds, but is still far longer than the non-service version. I also still had no .stat file in my sniffer directory. I did get a *.SVR file, which I never got before. So then I'm thinking, let's just make sure that I have the latest version of sniffer. I downloaded that, did the necessary renaming of the files and then started the service. NOW there is a *.persistent.stat file. However, the scan time is still at around 2 seconds. Average Scan times (based on average scan times of 5 emails each): Without sniffer service running: .033 seconds With sniffer service running: 2.244 seconds The *.persistent.stat file has the following contents: TicToc: 1122990610 Loop: 512 Poll: 445 Jobs: 34 Secs: 303 Msg/Min: 6.73267 Current-Load: 8.69565 Average-Load: 10.6371 Any suggestions? Thanks, Dan Horne This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Sniffer taking a long time?
I replied to an off-list message from Pete, but for completeness, I will repost it to the list. We can keep it on the list, Pete, if that does ya'. It looks like Pete is probably right in that the service is probably not loading correctly for some reason. There is no .stat file in my sniffer directory. Here are my responses to Pete's questions: > Can you please tell me the content of your .stat file. There is no .stat file in my sniffer directory. No file ending with .stat, either. > > Can you estimate the number of messages per minute that you are > processing? Fairly low volume, I guess, around 10 messages per minute. > Do you have a lot of extra files in your sniffer directory? Yes, there are tons of old *.FIN files, *.WRK files, *.XXX files, *.ERR files, and a few *.ABT files. However they are mostly old files. Sorting by date, I can see several *.FIN files, but they don't hang around long. There are several still there from each day though (I assume due to daily scheduled reboots according to the timestamp). The last occurrences of the other files by extension are: *.XXX - 7/24/2005 *.ERR - 4/27/2005 *.ABT - 2/4/2005 *.WRK - 12/14/2004 I assume it is ok to delete all these? > Does you have a lot of fragmentation in your file system? How do you > mitigate the fragmentation you do have? No, we defrag daily after hours using Diskeeper's smart scheduling. > This information will help. > > Thanks, > > _M > NP. I'm sure you saw my other posts to the list, but I'll recap. When I stop the service, processing time goes down to milliseconds. Reenabling the sniffer service (installed per the archived instructions using srvany.exe) causes the processing time to go back up into the minute per message range. I have the service disabled for now. We moved our Imail/Declude install off to a weaker machine a couple weeks ago in prep for replacing it with Suse Linux ES running postfix (and sniffer, of course) on the more powerful hardware. Because the current computer is not as powerful and has become backed up a few times, I was looking at ways to lower the CPU cost per message when I found this. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[2]: [sniffer] Sniffer taking a long time?
It seems that we'll need more information to understand what's happening here. In some cases, for example, a message might give up waiting for the persistent instance to handle it's message - then it will load the rulebase and process the message itself. I can't be completely sure based on the information you've provided so far, but it seems like that might be the case. That is - for the message in question - it waited close to a minute and then processed the message itself. The persistent instance should be publishing a .stat file if it is working properly. You can periodically "type" this file at the dos command line to see real-time data on the load and timing parameters. This information would help. Thanks, _M On Monday, August 1, 2005, 3:40:18 PM, Dan wrote: DH> More info: When I stop the Sniffer service, processing time goes to DH> milliseconds. Start the service back and it is back up to a minute. >> -Original Message- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of Dan Horne >> Sent: Monday, August 01, 2005 11:58 AM >> To: sniffer@SortMonster.com >> Subject: RE: [sniffer] Sniffer taking a long time? >> >> Here are the sniffer log entries for each of the messages, if >> that helps >> any: >> >> >> > 08/01/2005 11:32:51.747 Q40a201cc1a59 SNIFFER: External program >> > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe >> > mysnifferauthcode S:\imail\spool\D40a201cc1a59.SMD >> > 08/01/2005 11:33:46.751 Q40a201cc1a59 SNIFFER: External program >> > reports exit code of 61 >> >> 20050801153252D40a201cc1a59.SMD 70 20 >> Match 266707 >> 61343 358 50 >> 20050801153252D40a201cc1a59.SMD 70 20 >> Match 426427 >> 611915192950 >> 20050801153252D40a201cc1a59.SMD 70 20 >> Final 266707 >> 610 502050 >> >> >> > 08/01/2005 11:30:53.757 Q402b01b61a28 SNIFFER: External program >> > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe >> > mysnifferauthcode S:\imail\spool\D402b01b61a28.SMD >> > 08/01/2005 11:31:48.210 Q402b01b61a28 SNIFFER: External program >> > reports exit code of 52 >> >> 20050801153054D402b01b61a28.SMD 80 10 >> Match 372669 >> 522745286060 >> 20050801153054D402b01b61a28.SMD 80 10 >> Match 423177 >> 612695303660 >> 20050801153054D402b01b61a28.SMD 80 10 >> Match 372652 >> 612695313860 >> 20050801153054D402b01b61a28.SMD 80 10 >> Final 372669 >> 520 495260 >> >> > 08/01/2005 11:30:56.561 Q402a01cc1a27 SNIFFER: External program >> > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe >> > mysnifferauthcode S:\imail\spool\D402a01cc1a27.SMD >> > 08/01/2005 11:31:51.074 Q402a01cc1a27 SNIFFER: External program >> > reports exit code of 0 >> >> 20050801153056D402a01cc1a27.SMD 190 40 >> White 137999 >> 0 2256228544 >> 20050801153056D402a01cc1a27.SMD 190 40 >> Final 137999 >> 0 0 24419 44 >> >> This E-Mail came from the Message Sniffer mailing list. For >> information and (un)subscription instructions go to >> http://www.sortmonster.com/MessageSniffer/Help/Help.html >> DH> This E-Mail came from the Message Sniffer mailing list. For DH> information and (un)subscription instructions go to DH> http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Sniffer taking a long time?
What kind of box are you running Sniffer on and what is your CPU. I have found that most of the messages that get scanned on my system by Sniffer are done in under a second. Darrell Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Dan Horne writes: More info: When I stop the Sniffer service, processing time goes to milliseconds. Start the service back and it is back up to a minute. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Horne Sent: Monday, August 01, 2005 11:58 AM To: sniffer@SortMonster.com Subject: RE: [sniffer] Sniffer taking a long time? Here are the sniffer log entries for each of the messages, if that helps any: > 08/01/2005 11:32:51.747 Q40a201cc1a59 SNIFFER: External program > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > mysnifferauthcode S:\imail\spool\D40a201cc1a59.SMD > 08/01/2005 11:33:46.751 Q40a201cc1a59 SNIFFER: External program > reports exit code of 61 20050801153252 D40a201cc1a59.SMD 70 20 Match 266707 61 343 358 50 20050801153252 D40a201cc1a59.SMD 70 20 Match 426427 61 1915192950 20050801153252 D40a201cc1a59.SMD 70 20 Final 266707 61 0 502050 > 08/01/2005 11:30:53.757 Q402b01b61a28 SNIFFER: External program > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > mysnifferauthcode S:\imail\spool\D402b01b61a28.SMD > 08/01/2005 11:31:48.210 Q402b01b61a28 SNIFFER: External program > reports exit code of 52 20050801153054 D402b01b61a28.SMD 80 10 Match 372669 52 2745286060 20050801153054 D402b01b61a28.SMD 80 10 Match 423177 61 2695303660 20050801153054 D402b01b61a28.SMD 80 10 Match 372652 61 2695313860 20050801153054 D402b01b61a28.SMD 80 10 Final 372669 52 0 495260 > 08/01/2005 11:30:56.561 Q402a01cc1a27 SNIFFER: External program > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > mysnifferauthcode S:\imail\spool\D402a01cc1a27.SMD > 08/01/2005 11:31:51.074 Q402a01cc1a27 SNIFFER: External program > reports exit code of 0 20050801153056 D402a01cc1a27.SMD 190 40 White 137999 0 2256228544 20050801153056 D402a01cc1a27.SMD 190 40 Final 137999 0 0 24419 44 This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Sniffer taking a long time?
More info: When I stop the Sniffer service, processing time goes to milliseconds. Start the service back and it is back up to a minute. > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dan Horne > Sent: Monday, August 01, 2005 11:58 AM > To: sniffer@SortMonster.com > Subject: RE: [sniffer] Sniffer taking a long time? > > Here are the sniffer log entries for each of the messages, if > that helps > any: > > > > 08/01/2005 11:32:51.747 Q40a201cc1a59 SNIFFER: External program > > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > > mysnifferauthcode S:\imail\spool\D40a201cc1a59.SMD > > 08/01/2005 11:33:46.751 Q40a201cc1a59 SNIFFER: External program > > reports exit code of 61 > > 20050801153252D40a201cc1a59.SMD 70 20 > Match 266707 > 61343 358 50 > 20050801153252D40a201cc1a59.SMD 70 20 > Match 426427 > 611915192950 > 20050801153252D40a201cc1a59.SMD 70 20 > Final 266707 > 610 502050 > > > > 08/01/2005 11:30:53.757 Q402b01b61a28 SNIFFER: External program > > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > > mysnifferauthcode S:\imail\spool\D402b01b61a28.SMD > > 08/01/2005 11:31:48.210 Q402b01b61a28 SNIFFER: External program > > reports exit code of 52 > > 20050801153054D402b01b61a28.SMD 80 10 > Match 372669 > 522745286060 > 20050801153054D402b01b61a28.SMD 80 10 > Match 423177 > 612695303660 > 20050801153054D402b01b61a28.SMD 80 10 > Match 372652 > 612695313860 > 20050801153054D402b01b61a28.SMD 80 10 > Final 372669 > 520 495260 > > > 08/01/2005 11:30:56.561 Q402a01cc1a27 SNIFFER: External program > > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > > mysnifferauthcode S:\imail\spool\D402a01cc1a27.SMD > > 08/01/2005 11:31:51.074 Q402a01cc1a27 SNIFFER: External program > > reports exit code of 0 > > 20050801153056D402a01cc1a27.SMD 190 40 > White 137999 > 0 2256228544 > 20050801153056D402a01cc1a27.SMD 190 40 > Final 137999 > 0 0 24419 44 > > This E-Mail came from the Message Sniffer mailing list. For > information and (un)subscription instructions go to > http://www.sortmonster.com/MessageSniffer/Help/Help.html > This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Sniffer taking a long time?
Here are the sniffer log entries for each of the messages, if that helps any: > 08/01/2005 11:32:51.747 Q40a201cc1a59 SNIFFER: External program > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > mysnifferauthcode S:\imail\spool\D40a201cc1a59.SMD > 08/01/2005 11:33:46.751 Q40a201cc1a59 SNIFFER: External > program reports exit code of 61 20050801153252 D40a201cc1a59.SMD 70 20 Match 266707 61 343 358 50 20050801153252 D40a201cc1a59.SMD 70 20 Match 426427 61 1915192950 20050801153252 D40a201cc1a59.SMD 70 20 Final 266707 61 0 502050 > 08/01/2005 11:30:53.757 Q402b01b61a28 SNIFFER: External program > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > mysnifferauthcode S:\imail\spool\D402b01b61a28.SMD > 08/01/2005 11:31:48.210 Q402b01b61a28 SNIFFER: External > program reports exit code of 52 20050801153054 D402b01b61a28.SMD 80 10 Match 372669 52 2745286060 20050801153054 D402b01b61a28.SMD 80 10 Match 423177 61 2695303660 20050801153054 D402b01b61a28.SMD 80 10 Match 372652 61 2695313860 20050801153054 D402b01b61a28.SMD 80 10 Final 372669 52 0 495260 > 08/01/2005 11:30:56.561 Q402a01cc1a27 SNIFFER: External program > started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe > mysnifferauthcode S:\imail\spool\D402a01cc1a27.SMD > 08/01/2005 11:31:51.074 Q402a01cc1a27 SNIFFER: External > program reports exit code of 0 20050801153056 D402a01cc1a27.SMD 190 40 White 137999 0 2256228544 20050801153056 D402a01cc1a27.SMD 190 40 Final 137999 0 0 24419 44 This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Sniffer taking a long time?
OK, based on another thread on the Declude Junkmail list, I've taken a look at the Declude debug logs for a couple of messages. I am seeing this: 08/01/2005 11:32:51.747 Q40a201cc1a59 SNIFFER: External program started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe mysnifferauthcode S:\imail\spool\D40a201cc1a59.SMD 08/01/2005 11:33:46.751 Q40a201cc1a59 SNIFFER: External program reports exit code of 61 Am I reading this right (I must be) that this log snip shows sniffer taking almost a full minute to scan this message? Here are more: 08/01/2005 11:30:53.757 Q402b01b61a28 SNIFFER: External program started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe mysnifferauthcode S:\imail\spool\D402b01b61a28.SMD 08/01/2005 11:31:48.210 Q402b01b61a28 SNIFFER: External program reports exit code of 52 08/01/2005 11:30:56.561 Q402a01cc1a27 SNIFFER: External program started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe mysnifferauthcode S:\imail\spool\D402a01cc1a27.SMD 08/01/2005 11:31:51.074 Q402a01cc1a27 SNIFFER: External program reports exit code of 0 If so, I think I've found my bottleneck, and I guess I need help figuring out why it is taking so long to scan. These messages are always in this order (meaning the "program started" line is always right before the "program reports" line). This means (if I understand Declude's logging correctly) that Declude started sniffer, then sat back and waited almost a full minute for each email. I am running persistent sniffer (assuming I set it up correctly). Where can I look to find out why it is taking so long to scan? This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
