Re: Re[4]: [sniffer] Rash of false positives
This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard FarrisEthixs Online1.270.247. Office1.800.548.3877 Tech Support"Crossroads to a Cleaner Internet" - Original Message - From: Pete McNeil To: Darin Cox Sent: Tuesday, November 08, 2005 3:03 PM Subject: Re[4]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am. There were a number of different rules involved, and over 45 false positives in that time period. This is highly unusual -- I didn't remove many rules, and normally only one or two would be responsible. If you found that a large number of rules were responsible then something else happend and we need to look at that... I'd need to see your SNF logs from that period since the changes (removals anyway) in the rulebase were very small and unrelated - that just doesn't line up with your description. One thing does-- in the past if snf2check was not used to check a new download then a corrupted rulebase could cause SNF to produce erratic results... since snf2check has been in place we have not seen this. Is it possible that a bad rulebase file got pressed into service on your system? -- probably a look at the logs would help there too since this kind of failure is accompanied by very specific oddities in the logs. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Rash of false positives
We had this same thing happen. It has been happening more frequently recently and we are looking into disabling sniffer as it seems to be the culprit each time. John Moore 305 Spin From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Richard Farris Sent: Wednesday, November 09, 2005 11:38 AM To: sniffer@SortMonster.com Subject: Re: Re[4]: [sniffer] Rash of false positives This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard Farris Ethixs Online 1.270.247. Office 1.800.548.3877 Tech Support Crossroads to a Cleaner Internet - Original Message - From: Pete McNeil To: Darin Cox Sent: Tuesday, November 08, 2005 3:03 PM Subject: Re[4]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am. There were a number of different rules involved, and over 45 false positives in that time period. This is highly unusual -- I didn't remove many rules, and normally only one or two would be responsible. If you found that a large number of rules were responsible then something else happend and we need to look at that... I'd need to see your SNF logs from that period since the changes (removals anyway) in the rulebase were very small and unrelated - that just doesn't line up with your description. One thing does-- in the past if snf2check was not used to check a new download then a corrupted rulebase could cause SNF to produce erratic results... since snf2check has been in place we have not seen this. Is it possible that a bad rulebase file got pressed into service on your system? -- probably a look at the logs would help there too since this kind of failure is accompanied by very specific oddities in the logs. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[6]: [sniffer] Rash of false positives
This problem with Dr.Watson errors has been covered before on Declude's support list as well as ours. It's actually not SNF itselft that's causing the problem, but rather an undocumented heap in Windows that can run out of space and cause the next item to load to fail with a Dr. Watson error. SNF often is listed due to the way it is called by Declude which is called by IMail. There are some tuning parameters that can often mitigate the problem - I believe they are primarily concerned with the number of threads. Since the "mystery heap" is not documented there is no way to directly address the issue. The problem itself is documented (worth a google on the error code) as a number of programs run into this problem from time to time. Hope this helps, _M PS: If this is a different problem please send me the specific error code so I can research it. That said, since the code for SNF has not changed in some time it is highly unlikely that SNF would suddenly start causing DrWatson errors. The rulebase files are data - not executable code ;-) On Wednesday, November 9, 2005, 12:42:54 PM, John wrote: We had this same thing happen. It has been happening more frequently recently and we are looking into disablingsnifferas it seems to be the culprit each time. John Moore 305 Spin From:[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]On Behalf OfRichard Farris Sent:Wednesday, November 09, 2005 11:38 AM To:sniffer@SortMonster.com Subject:Re: Re[4]: [sniffer] Rash of false positives This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard Farris Ethixs Online 1.270.247. Office 1.800.548.3877 Tech Support "Crossroads to a Cleaner Internet" - Original Message - From:Pete McNeil To:Darin Cox Sent:Tuesday, November 08, 2005 3:03 PM Subject:Re[4]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am. There were a number of different rules involved, and over 45 false positives in that time period. This is highly unusual -- I didn't remove many rules, and normally only one or two would be responsible. If you found that a large number of rules were responsible then something else happend and we need to look at that... I'd need to see your SNF logs from that period since the changes (removals anyway) in the rulebase were very small and unrelated - that just doesn't line up with your description. One thing does-- in the past if snf2check was not used to check a new download then a corrupted rulebase could cause SNF to produce erratic results... since snf2check has been in place we have not seen this. Is it possible that a bad rulebase file got pressed into service on your system? -- probably a look at the logs would help there too since this kind of failure is accompanied by very specific oddities in the logs. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: Re[4]: [sniffer] Rash of false positives
Arecorrupted rulebase files the culprit? How do you update... and do you run snf2check on the updates? Just wondering if the rulebase file is theproblem, if the problemoccurs during the update, or if you are running into obscure errors with the EXE itself Darin. - Original Message - From: John Moore To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 12:42 PM Subject: RE: Re[4]: [sniffer] Rash of false positives We had this same thing happen. It has been happening more frequently recently and we are looking into disabling sniffer as it seems to be the culprit each time. John Moore305 Spin From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Richard FarrisSent: Wednesday, November 09, 2005 11:38 AMTo: sniffer@SortMonster.comSubject: Re: Re[4]: [sniffer] Rash of false positives This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard FarrisEthixs Online1.270.247. Office1.800.548.3877 Tech Support"Crossroads to a Cleaner Internet" - Original Message - From: Pete McNeil To: Darin Cox Sent: Tuesday, November 08, 2005 3:03 PM Subject: Re[4]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am. There were a number of different rules involved, and over 45 false positives in that time period. This is highly unusual -- I didn't remove many rules, and normally only one or two would be responsible. If you found that a large number of rules were responsible then something else happend and we need to look at that... I'd need to see your SNF logs from that period since the changes (removals anyway) in the rulebase were very small and unrelated - that just doesn't line up with your description. One thing does-- in the past if snf2check was not used to check a new download then a corrupted rulebase could cause SNF to produce erratic results... since snf2check has been in place we have not seen this. Is it possible that a bad rulebase file got pressed into service on your system? -- probably a look at the logs would help there too since this kind of failure is accompanied by very specific oddities in the logs. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Rash of false positives
We have not run snf2check on the updates. And it may be a coincidence or bad timing that sniffer appears to be the culprit. But we have stopped sniffer (commented out in the declude global.cfg) for an observed period of time and the mail never stops (and had never stopped before sniffer) and conversely, it only stops when sniffer is running. We have not gone the extra steps of putting sniffer in persistent mode. We are looking at moving the imail/declude/sniffer setup to a newer box with more resources. Currently on a dell 2450 dual 833 and 1 gig of ram and raid 5. Volume of email is less than 10,000 emails per day. J From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Wednesday, November 09, 2005 1:47 PM To: sniffer@SortMonster.com Subject: Re: Re[4]: [sniffer] Rash of false positives Arecorrupted rulebase files the culprit? How do you update... and do you run snf2check on the updates? Just wondering if the rulebase file is theproblem, if the problemoccurs during the update, or if you are running into obscure errors with the EXE itself Darin. - Original Message - From: John Moore To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 12:42 PM Subject: RE: Re[4]: [sniffer] Rash of false positives We had this same thing happen. It has been happening more frequently recently and we are looking into disabling sniffer as it seems to be the culprit each time. John Moore 305 Spin From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Richard Farris Sent: Wednesday, November 09, 2005 11:38 AM To: sniffer@SortMonster.com Subject: Re: Re[4]: [sniffer] Rash of false positives This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard Farris Ethixs Online 1.270.247. Office 1.800.548.3877 Tech Support Crossroads to a Cleaner Internet - Original Message - From: Pete McNeil To: Darin Cox Sent: Tuesday, November 08, 2005 3:03 PM Subject: Re[4]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am. There were a number of different rules involved, and over 45 false positives in that time period. This is highly unusual -- I didn't remove many rules, and normally only one or two would be responsible. If you found that a large number of rules were responsible then something else happend and we need to look at that... I'd need to see your SNF logs from that period since the changes (removals anyway) in the rulebase were very small and unrelated - that just doesn't line up with your description. One thing does-- in the past if snf2check was not used to check a new download then a corrupted rulebase could cause SNF to produce erratic results... since snf2check has been in place we have not seen this. Is it possible that a bad rulebase file got pressed into service on your system? -- probably a look at the logs would help there too since this kind of failure is accompanied by very specific oddities in the logs. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[6]: [sniffer] Rash of false positives
It is _VERY_ important to validate rulebase files with the snf2check utility. The snf2check utility tests the rulebase files in ways that the SNF scanning utility does not (for the sake of speed). If you don't check your downloads with the snf2check utility you run the risk of pressing a corrupt rulebase into service with unpredictable (but probably very bad) results. My $0.02 _M On Wednesday, November 9, 2005, 2:58:08 PM, John wrote: We havenotrunsnf2check on the updates. And it may be a coincidence or bad timing thatsnifferappears to be the culprit. But we have stoppedsniffer(commented out in thedecludeglobal.cfg) for an observed period of time and the mail never stops (and had never stopped beforesniffer) and conversely, it only stops whensnifferis running. We have not gone the extra steps of puttingsnifferin persistent mode. We are looking at moving theimail/declude/sniffersetup to a newer box with more resources. Currently on a dell 2450 dual 833 and 1 gig of ram and raid 5. Volume of email is less than 10,000 emails per day. J From:[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]On Behalf OfDarin Cox Sent:Wednesday, November 09, 2005 1:47 PM To:sniffer@SortMonster.com Subject:Re: Re[4]: [sniffer] Rash of false positives Are corrupted rulebase files the culprit? How do you update... and do you run snf2check on the updates? Just wondering if the rulebase file is the problem, if the problem occurs during the update, or if you are running into obscure errors with the EXE itself Darin. - Original Message - From:John Moore To:sniffer@SortMonster.com Sent:Wednesday, November 09, 2005 12:42 PM Subject:RE: Re[4]: [sniffer] Rash of false positives We had this same thing happen. It has been happening more frequently recently and we are looking into disabling sniffer as it seems to be the culprit each time. John Moore 305 Spin From:[EMAIL PROTECTED][mailto:[EMAIL PROTECTED]On Behalf OfRichard Farris Sent:Wednesday, November 09, 2005 11:38 AM To:sniffer@SortMonster.com Subject:Re: Re[4]: [sniffer] Rash of false positives This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard Farris Ethixs Online 1.270.247. Office 1.800.548.3877 Tech Support "Crossroads to a Cleaner Internet" - Original Message - From:Pete McNeil To:Darin Cox Sent:Tuesday, November 08, 2005 3:03 PM Subject:Re[4]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am. There were a number of different rules involved, and over 45 false positives in that time period. This is highly unusual -- I didn't remove many rules, and normally only one or two would be responsible. If you found that a large number of rules were responsible then something else happend and we need to look at that... I'd need to see your SNF logs from that period since the changes (removals anyway) in the rulebase were very small and unrelated - that just doesn't line up with your description. One thing does-- in the past if snf2check was not used to check a new download then a corrupted rulebase could cause SNF to produce erratic results... since snf2check has been in place we have not seen this. Is it possible that a bad rulebase file got pressed into service on your system? -- probably a look at the logs would help there too since this kind of failure is accompanied by very specific oddities in the logs. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Rash of false positives
John,
The mystery heap issue is a memory issue with Windows where it only
reserves so much memory for running things like Declude, Sniffer, other
external tests and your virus scanners. If you have something that is
hanging, running slowly, or taking too long, it can gobble up all of
the memory available to these launched processes and then result in
errors. Generally speaking, you can only get about 40 or so processes
of these types to run at one time before you could start seeing these
errors. Declude counts as one process, and often there is one other
process that Declude launches that goes to this count (external tests
and virus scanners are all run in serial so only one can be launched at
a time by a single Declude process). If you have something like a
virus scanner that crashes and then pops up a window on your next
login, this can count towards the number of open processes.
You can specify in Declude how many processes to run before Declude
starts dumping things into an overflow, either the overflow folder in
2.x and before, or something under proc in 3.x. If you create a file
called Declude.cfg and place in it "PROCESSES 20" that should protect
you from hitting the mystery heap's limitations unless something is
crashing and hanging. You might want to check Task Manager for
processes to verify if things are hanging since not everything will pop
up a window.
I believe that running Sniffer in persistent mode will help to
alleviate this condition, but it's only one part and if the mystery
heap is the cause, it might just cause the errors to be triggered on
other IMail launched processes including Declude.exe and your virus
scanners.
Matt
John Moore wrote:
We have not run snf2check on the
updates. And
it may be a coincidence or bad timing that sniffer
appears to be the culprit. But we have stopped sniffer
(commented out in the declude global.cfg)
for an observed period of time and the mail never stops (and had never
stopped
before sniffer) and conversely, it only
stops when sniffer is running.
We have not
gone the extra steps of
putting sniffer in persistent mode.
We are
looking at moving the imail/declude/sniffer
setup to a newer box with more
resources.
Currently on
a dell 2450 dual 833 and 1
gig of ram and raid 5. Volume of email is less than 10,000 emails per
day.
J
From:
[EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On
Behalf Of Darin Cox
Sent: Wednesday,
November 09, 2005
1:47 PM
To: sniffer@SortMonster.com
Subject: Re: Re[4]:
[sniffer] Rash
of false positives
Arecorrupted
rulebase files the culprit? How do you update... and do you run
snf2check on the updates?
Just wondering if
the rulebase file is theproblem, if the problemoccurs during the
update, or if you are running into obscure errors with the EXE
itself
Darin.
- Original
Message -
From: John Moore
To: sniffer@SortMonster.com
Sent: Wednesday,
November 09, 2005 12:42 PM
Subject: RE: Re[4]:
[sniffer] Rash of false positives
We had this
same thing happen.
It has been
happening more frequently
recently and we are looking into disabling sniffer as it seems to be
the
culprit each time.
John Moore
305 Spin
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On
Behalf Of Richard Farris
Sent: Wednesday,
November 09, 2005
11:38 AM
To: sniffer@SortMonster.com
Subject: Re: Re[4]:
[sniffer] Rash
of false positives
This
morning my server quit sending mail and my tech said the Dr.
Watson error on the server was my Sniffer file...I rebooted and thought
it was
OK but quit again..I had a lot of mail back logged...so I updated a new
rule
base but it did not seem to helpI reinstalled Imail and things seem
OK but
slow since there is such a back log of mailIf things don't get back
to
normal I will be back..
Richard Farris
Ethixs Online
1.270.247. Office
1.800.548.3877 Tech Support
"Crossroads to a Cleaner Internet"
-
Original Message -
From: Pete
McNeil
To: Darin
Cox
Sent:
Tuesday, November 08, 2005 3:03 PM
Subject: Re[4]:
[sniffer] Rash of false positives
On
Tuesday, November 8, 2005, 3:25:20
PM, Darin wrote:
Hi Pete,
There was a
consistent stream of false positives over the mentioned time period,
not just a blast at a particular time. They suddenly started at 5pm
(shortly after a 4:30pm rulesbase update), and were fairly evenly
spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails
came in between 11pm and 6am)...spanning 4 other rulebase updates at
8:40pm, 12am, 3am, and 6:20am. There were a number of different rules
involved, and over 45 false positives
RE: [sniffer] Rash of false positives
Matt, Thank you for your help and thorough explanation. I added the declude.cfg with the PROCESSES 20 We are running declude 2.06 and have the JM pro and AV standard. We will look into getting the persistent mode setup and see if that helps as well. Thanks, again. John From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Wednesday, November 09, 2005 4:49 PM To: sniffer@SortMonster.com Subject: Re: [sniffer] Rash of false positives John, The mystery heap issue is a memory issue with Windows where it only reserves so much memory for running things like Declude, Sniffer, other external tests and your virus scanners. If you have something that is hanging, running slowly, or taking too long, it can gobble up all of the memory available to these launched processes and then result in errors. Generally speaking, you can only get about 40 or so processes of these types to run at one time before you could start seeing these errors. Declude counts as one process, and often there is one other process that Declude launches that goes to this count (external tests and virus scanners are all run in serial so only one can be launched at a time by a single Declude process). If you have something like a virus scanner that crashes and then pops up a window on your next login, this can count towards the number of open processes. You can specify in Declude how many processes to run before Declude starts dumping things into an overflow, either the overflow folder in 2.x and before, or something under proc in 3.x. If you create a file called Declude.cfg and place in it PROCESSES 20 that should protect you from hitting the mystery heap's limitations unless something is crashing and hanging. You might want to check Task Manager for processes to verify if things are hanging since not everything will pop up a window. I believe that running Sniffer in persistent mode will help to alleviate this condition, but it's only one part and if the mystery heap is the cause, it might just cause the errors to be triggered on other IMail launched processes including Declude.exe and your virus scanners. Matt John Moore wrote: We have not run snf2check on the updates. And it may be a coincidence or bad timing that sniffer appears to be the culprit. But we have stopped sniffer (commented out in the declude global.cfg) for an observed period of time and the mail never stops (and had never stopped before sniffer) and conversely, it only stops when sniffer is running. We have not gone the extra steps of putting sniffer in persistent mode. We are looking at moving the imail/declude/sniffer setup to a newer box with more resources. Currently on a dell 2450 dual 833 and 1 gig of ram and raid 5. Volume of email is less than 10,000 emails per day. J From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Darin Cox Sent: Wednesday, November 09, 2005 1:47 PM To: sniffer@SortMonster.com Subject: Re: Re[4]: [sniffer] Rash of false positives Arecorrupted rulebase files the culprit? How do you update... and do you run snf2check on the updates? Just wondering if the rulebase file is theproblem, if the problemoccurs during the update, or if you are running into obscure errors with the EXE itself Darin. - Original Message - From: John Moore To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 12:42 PM Subject: RE: Re[4]: [sniffer] Rash of false positives We had this same thing happen. It has been happening more frequently recently and we are looking into disabling sniffer as it seems to be the culprit each time. John Moore 305 Spin From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Richard Farris Sent: Wednesday, November 09, 2005 11:38 AM To: sniffer@SortMonster.com Subject: Re: Re[4]: [sniffer] Rash of false positives This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard Farris Ethixs Online 1.270.247. Office 1.800.548.3877 Tech Support Crossroads to a Cleaner Internet - Original Message - From: Pete McNeil To: Darin Cox Sent: Tuesday, November 08, 2005 3:03 PM Subject: Re[4]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am
Re: [sniffer] Rash of false positives
i thought declude.cfg is for V 3.x Am I wrong ?is declude.cfg used with V 2.x ? - Original Message - From: John Moore To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 11:12 PM Subject: RE: [sniffer] Rash of false positives Matt, Thank you for your help and thorough explanation. I added the declude.cfg with the PROCESSES 20 We are running declude 2.06 and have the JM pro and AV standard. We will look into getting the persistent mode setup and see if that helps as well. Thanks, again. John From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MattSent: Wednesday, November 09, 2005 4:49 PMTo: sniffer@SortMonster.comSubject: Re: [sniffer] Rash of false positives John,The mystery heap issue is a memory issue with Windows where it only reserves so much memory for running things like Declude, Sniffer, other external tests and your virus scanners. If you have something that is hanging, running slowly, or taking too long, it can gobble up all of the memory available to these launched processes and then result in errors. Generally speaking, you can only get about 40 or so processes of these types to run at one time before you could start seeing these errors. Declude counts as one process, and often there is one other process that Declude launches that goes to this count (external tests and virus scanners are all run in serial so only one can be launched at a time by a single Declude process). If you have something like a virus scanner that crashes and then pops up a window on your next login, this can count towards the number of open processes.You can specify in Declude how many processes to run before Declude starts dumping things into an overflow, either the overflow folder in 2.x and before, or something under proc in 3.x. If you create a file called Declude.cfg and place in it "PROCESSES 20" that should protect you from hitting the mystery heap's limitations unless something is crashing and hanging. You might want to check Task Manager for processes to verify if things are hanging since not everything will pop up a window.I believe that running Sniffer in persistent mode will help to alleviate this condition, but it's only one part and if the mystery heap is the cause, it might just cause the errors to be triggered on other IMail launched processes including Declude.exe and your virus scanners.MattJohn Moore wrote: We have not run snf2check on the updates. And it may be a coincidence or bad timing that sniffer appears to be the culprit. But we have stopped sniffer (commented out in the declude global.cfg) for an observed period of time and the mail never stops (and had never stopped before sniffer) and conversely, it only stops when sniffer is running. We have not gone the extra steps of putting sniffer in persistent mode. We are looking at moving the imail/declude/sniffer setup to a newer box with more resources. Currently on a dell 2450 dual 833 and 1 gig of ram and raid 5. Volume of email is less than 10,000 emails per day. J From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Darin CoxSent: Wednesday, November 09, 2005 1:47 PMTo: sniffer@SortMonster.comSubject: Re: Re[4]: [sniffer] Rash of false positives Arecorrupted rulebase files the culprit? How do you update... and do you run snf2check on the updates? Just wondering if the rulebase file is theproblem, if the problemoccurs during the update, or if you are running into obscure errors with the EXE itself Darin. - Original Message - From: John Moore To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 12:42 PM Subject: RE: Re[4]: [sniffer] Rash of false positives We had this same thing happen. It has been happening more frequently recently and we are looking into disabling sniffer as it seems to be the culprit each time. John Moore305 Spin From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Richard FarrisSent: Wednesday, November 09, 2005 11:38 AMTo: sniffer@SortMonster.comSubject: Re: Re[4]: [sniffer] Rash of false positives This morning my server quit sending mail and my tech said the Dr. Watson error on the server was my Sniffer file...I rebooted and thought it was OK but quit again..I had a lot of mail back logged...so I updated a new rule base but it did not seem to helpI reinstalled Imail and things seem OK but slow since there is such a back log of mailIf things don't get back to normal I will be back.. Richard FarrisEthixs Online1.270.247. Office1.800.548.3877 Tech Support"Crossroads to a Clea
Re: [sniffer] Rash of false positives
It is used in both versions for different
things.
Darrell
---Check out http://www.invariantsystems.com for
utilities for Declude, mxGuard,and Imail. IMail Queue Monitoring,
Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and
Log Parsers.
- Original Message -
From:
Serge
To: sniffer@SortMonster.com
Sent: Wednesday, November 09, 2005 9:27
PM
Subject: Re: [sniffer] Rash of false
positives
i thought declude.cfg is for V 3.x
Am I wrong ?is declude.cfg used with V 2.x
?
- Original Message -
From:
John Moore
To: sniffer@SortMonster.com
Sent: Wednesday, November 09, 2005
11:12 PM
Subject: RE: [sniffer] Rash of false
positives
Matt,
Thank you for your
help and thorough explanation. I added the declude.cfg with the PROCESSES
20
We are running
declude 2.06 and have the JM pro and AV
standard.
We will look into
getting the persistent mode setup and see if that helps as
well.
Thanks,
again.
John
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On
Behalf Of MattSent: Wednesday, November 09, 2005 4:49
PMTo: sniffer@SortMonster.comSubject: Re: [sniffer] Rash of false
positives
John,The mystery heap issue is a memory
issue with Windows where it only reserves so much memory for running things
like Declude, Sniffer, other external tests and your virus scanners.
If you have something that is hanging, running slowly, or taking too long,
it can gobble up all of the memory available to these launched processes and
then result in errors. Generally speaking, you can only get about 40
or so processes of these types to run at one time before you could start
seeing these errors. Declude counts as one process, and often there is
one other process that Declude launches that goes to this count (external
tests and virus scanners are all run in serial so only one can be launched
at a time by a single Declude process). If you have something like a
virus scanner that crashes and then pops up a window on your next login,
this can count towards the number of open processes.You can specify
in Declude how many processes to run before Declude starts dumping things
into an overflow, either the overflow folder in 2.x and before, or something
under proc in 3.x. If you create a file called Declude.cfg and place
in it "PROCESSES 20" that should protect you from hitting the
mystery heap's limitations unless something is crashing and hanging.
You might want to check Task Manager for processes to verify if things are
hanging since not everything will pop up a window.I believe that
running Sniffer in persistent mode will help to alleviate this condition,
but it's only one part and if the mystery heap is the cause, it might just
cause the errors to be triggered on other IMail launched processes including
Declude.exe and your virus scanners.MattJohn Moore
wrote:
We have not run snf2check on the
updates. And it may be a coincidence or bad timing that sniffer appears to
be the culprit. But we have stopped sniffer (commented out in the declude
global.cfg) for an observed period of time and the mail never stops (and had
never stopped before sniffer) and conversely, it only stops when sniffer is
running.
We have not gone
the extra steps of putting sniffer in persistent
mode.
We are looking at
moving the imail/declude/sniffer setup to a newer box with more
resources.
Currently on a dell
2450 dual 833 and 1 gig of ram and raid 5. Volume of email is less than
10,000 emails per day.
J
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]
On Behalf Of Darin
CoxSent: Wednesday,
November 09, 2005 1:47 PMTo: sniffer@SortMonster.comSubject: Re: Re[4]: [sniffer] Rash of
false positives
Arecorrupted
rulebase files the culprit? How do you update... and do you run
snf2check on the updates?
Just wondering if
the rulebase file is theproblem, if the problemoccurs during the
update, or if you are running into obscure errors with the EXE
itself
Darin.
- Original
Message -
From: John Moore
To: sniffer@SortMonster.com
Sent:
Wednesday, November 09, 2005 12:42
PM
Subject: RE:
Re[4]: [sniffer] Rash of false
positives
We had this same
thing happen.
It has been
happening more frequently recently and we are looking into disabling sniffer
as it seems to be the culprit each
time.
John Moore305
[sniffer] Rash of false positives
Hi Pete, What's going on over there? We had somewhere between 5 and 10 times the usual number of Sniffer false positives this morning. They are across the board, so it's not just one rule that's catching them, or a particular set of senders or receivers. Hopefully you can get it under control soon. It would also be extremely helpful if you could speed up the false positive processing. Lately it seems to take 2-4 days for the rules to be adjusted, which usually means more of the same are caught and submitted over that time. I believe speeding up that process would result in fewer to process all around. Thanks, Darin.
Re: [sniffer] Rash of false positives
Dear Darin, Thanks for the heads up. It's going to take me about 45 minutes to check the 9000 messages that were blocked by Sniffer last night, but I'll let you know if we experienced the same thing. Michael SteinComputer House www.computerhouse.com - Original Message - From: Darin Cox To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 8:45 AM Subject: [sniffer] Rash of false positives Hi Pete, What's going on over there? We had somewhere between 5 and 10 times the usual number of Sniffer false positives this morning. They are across the board, so it's not just one rule that's catching them, or a particular set of senders or receivers. Hopefully you can get it under control soon. It would also be extremely helpful if you could speed up the false positive processing. Lately it seems to take 2-4 days for the rules to be adjusted, which usually means more of the same are caught and submitted over that time. I believe speeding up that process would result in fewer to process all around. Thanks, Darin.
Re: [sniffer] Rash of false positives
We're seeing a continual stream of false positives. It's taking all of our time just to keep up with it at the moment. If something isn't done soon, we're going to have to disable sniffer. Darin. - Original Message - From: Computer House Support To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 9:34 AM Subject: Re: [sniffer] Rash of false positives Dear Darin, Thanks for the heads up. It's going to take me about 45 minutes to check the 9000 messages that were blocked by Sniffer last night, but I'll let you know if we experienced the same thing. Michael SteinComputer House www.computerhouse.com - Original Message - From: Darin Cox To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 8:45 AM Subject: [sniffer] Rash of false positives Hi Pete, What's going on over there? We had somewhere between 5 and 10 times the usual number of Sniffer false positives this morning. They are across the board, so it's not just one rule that's catching them, or a particular set of senders or receivers. Hopefully you can get it under control soon. It would also be extremely helpful if you could speed up the false positive processing. Lately it seems to take 2-4 days for the rules to be adjusted, which usually means more of the same are caught and submitted over that time. I believe speeding up that process would result in fewer to process all around. Thanks, Darin.
Re: [sniffer] Rash of false positives
After reviewing all the blocked messages for the past 2 days on 2 different servers, I found no false positives. Do you happen to have an old rule base from several days again ? If so, try that to see if it temporarily resolves the false positives. -Original Message-From: "Darin Cox" [EMAIL PROTECTED]To: sniffer@SortMonster.comDate: Tue, 8 Nov 2005 08:45:39 -0500 Subject: [sniffer] Rash of false positives Hi Pete, What's going on over there? We had somewhere between 5 and 10 times the usual number of Sniffer false positives this morning. They are across the board, so it's not just one rule that's catching them, or a particular set of senders or receivers. Hopefully you can get it under control soon. It would also be extremely helpful if you could speed up the false positive processing. Lately it seems to take 2-4 days for the rules to be adjusted, which usually means more of the same are caught and submitted over that time. I believe speeding up that process would result in fewer to process all around. Thanks, Darin.
Re: [sniffer] Rash of false positives
No, we automatically update with every notification of a new rulebase. Looking further, they started just before 5pm ET yesterday. So far, it's about 10 times the usual number of Sniffer false positives. We've sent quite a few this morning to false (at) for processing. Darin. - Original Message - From: Paul Lushinsky To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 10:10 AM Subject: Re: [sniffer] Rash of false positives After reviewing all the blocked messages for the past 2 days on 2 different servers, I found no false positives. Do you happen to have an old rule base from several days again ? If so, try that to see if it temporarily resolves the false positives. -Original Message-From: "Darin Cox" [EMAIL PROTECTED]To: sniffer@SortMonster.comDate: Tue, 8 Nov 2005 08:45:39 -0500Subject: [sniffer] Rash of false positives Hi Pete, What's going on over there? We had somewhere between 5 and 10 times the usual number of Sniffer false positives this morning. They are across the board, so it's not just one rule that's catching them, or a particular set of senders or receivers. Hopefully you can get it under control soon. It would also be extremely helpful if you could speed up the false positive processing. Lately it seems to take 2-4 days for the rules to be adjusted, which usually means more of the same are caught and submitted over that time. I believe speeding up that process would result in fewer to process all around. Thanks, Darin.
Re: [sniffer] Rash of false positives
I don't know if I would call it a rash, but over
the last week, I've submitted about 30 false positives. That's far more than
average.
I've developed a feeling that Message Sniffer has
become "too tight".
- Original Message -
From:
Darin Cox
To: sniffer@SortMonster.com
Sent: Tuesday, November 08, 2005 8:54
AM
Subject: Re: [sniffer] Rash of false
positives
We're seeing a continual stream of false
positives. It's taking all of our time just to keep up with it at the
moment. If something isn't done soon, we're going to have to disable
sniffer.
Darin.
- Original Message -
From: Computer House Support
To: sniffer@SortMonster.com
Sent: Tuesday, November 08, 2005 9:34 AM
Subject: Re: [sniffer] Rash of false positives
Dear Darin,
Thanks for the heads up. It's going to take me
about 45 minutes to check the 9000 messages that were blocked by Sniffer last
night, but I'll let you know if we experienced the same thing.
Michael SteinComputer House
www.computerhouse.com
- Original Message -
From:
Darin Cox
To: sniffer@SortMonster.com
Sent: Tuesday, November 08, 2005 8:45
AM
Subject: [sniffer] Rash of false
positives
Hi Pete,
What's going on over there? We had
somewhere between 5 and 10 times the usual number of Sniffer false positives
this morning. They are across the board, so it's not just one rule
that's catching them, or a particular set of senders or
receivers.
Hopefully you can get it under control
soon.
It would also be extremely helpful if you could
speed up the false positive processing. Lately it seems to take 2-4
days for the rules to be adjusted, which usually means more of the same are
caught and submitted over that time. I believe speeding up that
process would result in fewer to process all around.
Thanks,
Darin.
Re: [sniffer] Rash of false positives
I've submitted about 45 so far this morning. I normally submit at most a half dozen each morning. Darin. - Original Message - From: Darrell ([EMAIL PROTECTED]) [EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 10:19 AM Subject: Re: [sniffer] Rash of false positives I too have had to submit a lot more false positives lately. I also second that false positive processing seems to be a lot slower than previously. Darrell Check out http://www.invariantsystems.com for utilities for Declude, mxGuard, And Imail. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Scott Fisher writes: I don't know if I would call it a rash, but over the last week, I've submitted about 30 false positives. That's far more than average. I've developed a feeling that Message Sniffer has become too tight. - Original Message - From: Darin Cox To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 8:54 AM Subject: Re: [sniffer] Rash of false positives We're seeing a continual stream of false positives. It's taking all of our time just to keep up with it at the moment. If something isn't done soon, we're going to have to disable sniffer. Darin. - Original Message - From: Computer House Support To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 9:34 AM Subject: Re: [sniffer] Rash of false positives Dear Darin, Thanks for the heads up. It's going to take me about 45 minutes to check the 9000 messages that were blocked by Sniffer last night, but I'll let you know if we experienced the same thing. Michael Stein Computer House www.computerhouse.com - Original Message - From: Darin Cox To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 8:45 AM Subject: [sniffer] Rash of false positives Hi Pete, What's going on over there? We had somewhere between 5 and 10 times the usual number of Sniffer false positives this morning. They are across the board, so it's not just one rule that's catching them, or a particular set of senders or receivers. Hopefully you can get it under control soon. It would also be extremely helpful if you could speed up the false positive processing. Lately it seems to take 2-4 days for the rules to be adjusted, which usually means more of the same are caught and submitted over that time. I believe speeding up that process would result in fewer to process all around. Thanks, Darin. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Rash of false positives
Hi Pete, The rash of false positivesseems to have stopped with the last sniffer rulebase update at 10am ET. It had started with a rulebase update at 4:30pm ET yesterday, and continued through the updates at 8:40pm, 12am, 3am, and 6:20am today. I'd still like to know what happened, and how we can avoid it in the future. Thanks, Darin. - Original Message - From: Darin Cox To: sniffer@SortMonster.com Sent: Tuesday, November 08, 2005 8:45 AM Subject: [sniffer] Rash of false positives Hi Pete, What's going on over there? We had somewhere between 5 and 10 times the usual number of Sniffer false positives this morning. They are across the board, so it's not just one rule that's catching them, or a particular set of senders or receivers. Hopefully you can get it under control soon. It would also be extremely helpful if you could speed up the false positive processing. Lately it seems to take 2-4 days for the rules to be adjusted, which usually means more of the same are caught and submitted over that time. I believe speeding up that process would result in fewer to process all around. Thanks, Darin.
Re[2]: [sniffer] Rash of false positives
On Tuesday, November 8, 2005, 11:02:09 AM, Darin wrote: Hi Pete, The rash of false positives seems to have stopped with the last sniffer rulebase update at 10am ET. It had started with a rulebase update at 4:30pm ET yesterday, and continued through the updates at 8:40pm, 12am, 3am, and 6:20am today. I'd still like to know what happened, and how we can avoid it in the future. I've been bound up in some performance tuning today so I've not had a chance to follow this thread until now. When I first looked in on it I scanned the false positive submissions and almost none of them matched any active rules. I know that a couple of rules were pulled out after review last night late .. they had been picked up by some FPs in SURBL others that matched up with spamtrap submissions. It's possible that these are what you experienced. I won't know unless you can give me some log entries to go with those messages since those entries will tell me the rule IDs. As for having it happen again - that's very unlikely since ever time we pull a rule out due to FPs or potential FPs (the rules that were pulled had not caused any FPs yet but were expected to... one was rr.com IIRC, it was pulled only a couple hours after it's creation). A lot of things have to go wrong to cause an FP problem like you are reporting. Please look up our rule-panic procedure which is designed to mitigate these problems immediately for you if they happen: http://www.sortmonster.com/MessageSniffer/Help/FalsePositivesHelp.html#RulePanic We can't guarantee that rule-panics won't happen, but we can make them exceedingly rare and non-repeatable. I will be processing your FP submissions shortly. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[4]: [sniffer] Rash of false positives
On Tuesday, November 8, 2005, 3:25:20 PM, Darin wrote: Hi Pete, There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time. They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am. There were a number of different rules involved, and over 45 false positives in that time period. This is highly unusual -- I didn't remove many rules, and normally only one or two would be responsible. If you found that a large number of rules were responsible then something else happend and we need to look at that... I'd need to see your SNF logs from that period since the changes (removals anyway) in the rulebase were very small and unrelated - that just doesn't line up with your description. One thing does-- in the past if snf2check was not used to check a new download then a corrupted rulebase could cause SNF to produce erratic results... since snf2check has been in place we have not seen this. Is it possible that a bad rulebase file got pressed into service on your system? -- probably a look at the logs would help there too since this kind of failure is accompanied by very specific oddities in the logs. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
