Eric,

I'm guessing based on what you were seeing, that it was unrelated to what I was seeing. Sniffer never actually died, it just got over 100 times slower, and 1/8th of the time it timed out. This never happened before 5/22, and this same server has been there for years, and the same installation of Sniffer for 2 years or so. I would think that if the issue was I/O (under normal conditions), it would have happened before 5/22 as there were clearly bursty periods often enough that my own traffic didn't change dramatically enough so that it happened 4 to 5 times in one month.


The server itself could have some issues that could be causing this. Maybe the file system is screwy, or Windows itself, or memory errors, or whatever.

Matt


On 6/28/2013 2:12 PM, E. H. (Eric) Fletcher wrote:
Matt:

I mentioned in a previous post that we had experienced something similar at
about that time and resolved it a day or so later by re-installing sniffer
when service restarts, reboots and some basic troubleshooting did not give
us the results we needed.  At this point that still seems to have been
effective (about 5 days now).

At the time, we did move things around to see whether it was related to the
number of items in the queue or anywhere else within the structure of the
mail system and found it made no difference. A single item arriving in an
empty Queue was still not processed.   CPU utilization was modest (single
digit across 4 cores) and disk I/O was lighter than usual as it took place
over a weekend.  Memory utilization was a little higher than I'd like to
see, we are addressing that now.

Following a suggestion from another ISP, we moved the spool folders onto a
RAM drive a couple of months ago.  That has worked well for us, we did rule
it out as the source of the problem by moving back onto the conventional
hard disk during the last part of the troubleshooting and for the first hour
or two following the reload.  We are processing on the Ramdisk now and have
been for over 4 days again.

For what it's worth . . .

Eric


-----Original Message-----
From: Message Sniffer Community [mailto:sniffer@sortmonster.com] On Behalf
Of Matt
Sent: Friday, June 28, 2013 10:32 AM
To: Message Sniffer Community
Subject: [sniffer] Re: Slow processing times, errors

Pete,

Just after the restart of the Sniffer service, times dropped back down into
the ms from 30+ seconds before, so what I am saying is that if I/O was the
issue, it was merely the trigger for something that put the service in a bad
state when it started.  I/O issues are not persistent, but could happen from
time to time I'm sure. Restarting Sniffer with a backlog of 2,500 messages
and normal peak traffic will not re-trigger the condition, and I press
Declude to run up to 300 messages at a time in situations like that, and the
CPU's are pegged until the backlog clears.  In the past, I restarted the
whole system, not knowing why it worked.  During normal peak times (without
bursts), the Declude is processing about 125 messages at a time which take
an average of 6 seconds to fully process, and therefore Sniffer is probably
handling only about 10 messages at a time (at peak).

Since 5/22 I have seen 4 or 5 different events like this, and I confirmed
that they are all present in the SNFclient.exe.err log.

Matt



On 6/28/2013 12:41 PM, Pete McNeil wrote:
On 2013-06-28 12:10, Matt wrote:
I am looking to retool presently just because it's time.  So if you
are convinced that this is due to low resources, don't concern
yourself with it.
Ok. It makes sense that the ~200 messages all at once could have
happend at the restart. SNFClient will keep trying for 30-90 seconds
before it gives up and spits out it's error file. That's where your
delays are coming from. SNF itself was clocking only about 100-800ms
for all of the scans.

The error result you report is exactly the one sent by SNF -- that it
was unable to open the file.

I am very sure this is resource related -- your scans should not be
taking the amount of time they are and I suspect most of that time is
eaten up trying to get to the files. The occasional errors of the same
time are a good hint that IO is to blame.

The new spam that we've seen often includes large messages -- so
that's going to put a higher load on IO resources -- I'll bet that the
increased volume and large message sizes are pushing IO over the edge
or at least very close to it.

Best,

_M



#############################################################
This message is sent to you because you are subscribed to
   the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and
related email topics.
For More information see http://www.armresearch.com To unsubscribe, E-mail
to: <sniffer-...@sortmonster.com> To switch to the DIGEST mode, E-mail to
<sniffer-dig...@sortmonster.com> To switch to the INDEX mode, E-mail to
<sniffer-in...@sortmonster.com> Send administrative queries to
<sniffer-requ...@sortmonster.com>



#############################################################
This message is sent to you because you are subscribed to
   the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>





#############################################################
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>

Reply via email to