RE: Re[2]: [sniffer] Sniffer taking a long time?

2005-08-03 Thread Dan Horne
Thanks, I will do that. 

> -Original Message-
> From: [EMAIL PROTECTED] 
> [mailto:[EMAIL PROTECTED] On Behalf Of Colbeck, Andrew
> Sent: Wednesday, August 03, 2005 3:17 AM
> To: sniffer@SortMonster.com
> Subject: RE: Re[2]: [sniffer] Sniffer taking a long time?
> 
> > So basically, what you are saying is that my volume is 
> really too low 
> > to take advantage of the persistent sniffer (and such may actually 
> > decrease my performance), and I should stick with the non-service 
> > version.  Is that right?  That is about what I thought (without the 
> > details of how sniffer works, I just wanted to be sure).
> 
> Well, Dan, for the inevitable rush of traffic, I'd stick with 
> the persistent sniffer implementation now that you have it working.
> 
> If the 2 second wait time galls you, then use your **.cfg 
> file and specify the
> 
> MaxPollTime: 500
> 
> value at 500 ms or whatever you'd like your maximum wait time 
> to be instead of 2 seconds (2000 ms).
> 
> Andrew 8)
> 
> 
> 
> 
> This E-Mail came from the Message Sniffer mailing list. For 
> information and (un)subscription instructions go to 
> http://www.sortmonster.com/MessageSniffer/Help/Help.html
> 

This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html


RE: Re[2]: [sniffer] Sniffer taking a long time?

2005-08-03 Thread Colbeck, Andrew
> So basically, what you are saying is that my volume is really 
> too low to take advantage of the persistent sniffer (and such 
> may actually decrease my performance), and I should stick 
> with the non-service version.  Is that right?  That is about 
> what I thought (without the details of how sniffer works, I 
> just wanted to be sure).

Well, Dan, for the inevitable rush of traffic, I'd stick with the
persistent sniffer implementation now that you have it working.

If the 2 second wait time galls you, then use your **.cfg file and
specify the

MaxPollTime: 500

value at 500 ms or whatever you'd like your maximum wait time to be
instead of 2 seconds (2000 ms).

Andrew 8)




This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html


RE: Re[2]: [sniffer] Sniffer taking a long time?

2005-08-02 Thread Dan Horne
So basically, what you are saying is that my volume is really too low to take 
advantage of the persistent sniffer (and such may actually decrease my 
performance), and I should stick with the non-service version.  Is that right?  
That is about what I thought (without the details of how sniffer works, I just 
wanted to be sure).

Thanks, Pete.

Dan Horne

> -Original Message-
> From: [EMAIL PROTECTED] 
> [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil
> Sent: Tuesday, August 02, 2005 4:09 PM
> To: Dan Horne
> Subject: Re[2]: [sniffer] Sniffer taking a long time?
> 
> After following through all of this and looking at the .stat 
> file, I think I see what's going on.
> 
> Now that it is running and producing a .stat file, the flow 
> rate is very low. According to the stat data, about 6 msgs / minute.
> 
> Note the poll and loop times are in the 450 - 550 ms range.
> 
> SNF with the persistent engine is built for high throughput, 
> but it's also built to play nice.
> 
> The maximum poll time gets up to 2 seconds or so (sound familiar?)
> 
> If there are no messages for a while, then everything slows 
> down until the first message goes through. For that first 
> message, the SNF client will probably wait about 2 seconds 
> before looking for it's result because that's what the stat 
> file will tell it to do.
> 
> Since the next message probably won't come around for a few 
> seconds, that next message will probably wait about 2 seconds also.
> 
> If you were doing 6 messages a second then all of the times 
> would be much lower and so would the individual delays.
> 
> When you turn off the persistent instance, each new message 
> causes a client to look and see if there are any other peers 
> acting a servers... Since the messages are far and few 
> between, the client will elect to be a server (momentarily), 
> will find no work but it's own, will process it's own message 
> and leave. -- This is the automatic peer-server mode. It will 
> always work like this unless more than one message is being 
> processed at the same moment.
> 
> In peer-server mode, since there is nothing else going on and 
> no persistent instance to coordinate the operations, each 
> message will get processed as fast as the rulebase can be 
> loaded and then the program will drop.
> 
> When the persistent instance is introduced, it sets the pace 
> - and sicne there are no other messages, each client will 
> wait about 2 seconds (or half a second or so with the .stat 
> file contents you show) before it begins looking for it's results.
> 
> The server instance will also wait a bit before looking for 
> new jobs so that the file system isn't constantly being scanned.
> 
> Of course, if a burst of messages come through then the 
> pacing will speed up as much as necessary to keep up with the volume.
> 
> Hope this helps,
> 
> _M
> 
> On Tuesday, August 2, 2005, 3:38:52 PM, Dan wrote:
> 
> DH> No, I followed your instructions exactly (and not for the first 
> DH> time).  I didn't add those extra values until today.  Prior to  
> DH> adding the AppDirectory value, the service was taking a minute to 
> DH> scan emails;  after adding it the scan time went to around 2 
> DH> seconds.  I can't get it any  lower than that.  Initially 
> mine was 
> DH> set up exactly as you said, with only  "Application" 
> containing the 
> DH> path, authcode and persistent.  Today after  hearing no 
> suggestions 
> DH> from the list, and based on recent list messages 
> mentioning the home 
> DH> directory for the service, I looked at the srvany.exe 
> doco  to find 
> DH> out how to give it a home directory.
> DH> That's when I added  AppDirectory.  I also saw and added 
> DH> AppParameters at the same time and  added those as well, 
> though they 
> DH> seem not to be needed.
> DH>  
> DH> Prior to adding the AppDirectory value, I never got any 
> .stat file 
> DH> or any .SVR file in my sniffer dir.  After adding that value and  
> DH> starting the service those files appeared.
> DH>  
> DH>  
> 
> 
> DH> From: [EMAIL PROTECTED]
> DH> [mailto:[EMAIL PROTECTED] On  Behalf Of Matt
> DH> Sent: Tuesday, August 02, 2005 3:24  PM
> DH> To: sniffer@SortMonster.com
> DH> Subject: Re: [sniffer]  Sniffer taking a long time?
> 
> 
>   
> 
> DH> Dan,
> 
> DH> There is no AppDirectory value on my servereither.  The
> DH> Parameters key has only one value under it besides Default   
> DH> which is "Application", and it contains exactly what I provided
> DH> below. Could it be that you tried to hard to get everything
> DH> right by tweaking theseadditional keys?
> 
> DH> Something else.  Did you make sure that theSniffer
> DH> service that you created was started?  No doubt it will work if   
> DH> you follow those directions to a T, and there aren't any issues
> DH> with yourserver apart from this.
> 
> DH> Matt
> 
> 
> 
> DH> Dan Horne wrote: 
>   
> 
> 
> DH>   I removed the AppParameters value and put the authcode 
> DH> an

RE: Re[2]: [sniffer] Sniffer taking a long time?

2005-08-01 Thread Dan Horne
I replied to an off-list message from Pete, but for completeness, I will
repost it to the list.  We can keep it on the list, Pete, if that does
ya'.  It looks like Pete is probably right in that the service is
probably not loading correctly for some reason.  There is no .stat file
in my sniffer directory.  Here are my responses to Pete's questions:

> Can you please tell me the content of your .stat file.

There is no .stat file in my sniffer directory.  No file ending with
.stat, either.

> 
> Can you estimate the number of messages per minute that you are 
> processing?

Fairly low volume, I guess, around 10 messages per minute.
 
> Do you have a lot of extra files in your sniffer directory?

Yes, there are tons of old *.FIN files, *.WRK files, *.XXX files, *.ERR
files, and a few *.ABT files.  However they are mostly old files.
Sorting by date, I can see several *.FIN files, but they don't hang
around long.  There are several still there from each day though (I
assume due to daily scheduled reboots according to the timestamp).  The
last occurrences of the other files by extension are:

*.XXX - 7/24/2005
*.ERR - 4/27/2005
*.ABT - 2/4/2005
*.WRK - 12/14/2004

I assume it is ok to delete all these?

> Does you have a lot of fragmentation in your file system? How do you 
> mitigate the fragmentation you do have?

No, we defrag daily after hours using Diskeeper's smart scheduling.

> This information will help.
> 
> Thanks,
> 
> _M
> 

NP.  I'm sure you saw my other posts to the list, but I'll recap.  When
I stop the service, processing time goes down to milliseconds.
Reenabling the sniffer service (installed per the archived instructions
using srvany.exe) causes the processing time to go back up into the
minute per message range.  I have the service disabled for now.  We
moved our Imail/Declude install off to a weaker machine a couple weeks
ago in prep for replacing it with Suse Linux ES running postfix (and
sniffer, of course) on the more powerful hardware.  Because the current
computer is not as powerful and has become backed up a few times, I was
looking at ways to lower the CPU cost per message when I found this. 


This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html