[sniffer] Anyone on the list using postfix?
Please let me know, we might be able to help each other... -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: snfilter - linux - postfix
Hi Pete, Just for information, we renamed the msg folder again today, and again SNFClient.exe.err only state: Could Not Connect! /etc/init.d/snfilter stop + /etc/init.d/snfilter start helped. Hello Pi-Web, Sunday, January 27, 2008, 1:16:08 PM, you wrote: Sorry, I might not have been clear. It is on Linux with postfix. I should have picked that out of the path. ;-) Yes stop/start of the service did solve the problem. Before start/stop pstree showed 14*SNFserver.exe SNFClient.exe.err only state: Could Not Connect! Last x.200801??.log.xml ends with: i u='20080125234317' context='--INITIALIZING--' code='0' text='Success'/ Rest seems normal. That also seems normal for a start-up. So I have no clue why it stoped. This is unusual. I've repeatedly had SNFServer run for weeks and months on various platforms -- almost without exception it only stops when I tell it to stop (including earlier test versions). If you come across any new info please let me know. If there is a bug I want it gone ;-) Thanks! _M -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: snfilter - linux - postfix
Sorry, I might not have been clear. It is on Linux with postfix. Yes stop/start of the service did solve the problem. Before start/stop pstree showed 14*SNFserver.exe SNFClient.exe.err only state: Could Not Connect! Last x.200801??.log.xml ends with: i u='20080125234317' context='--INITIALIZING--' code='0' text='Success'/ Rest seems normal. So I have no clue why it stoped. Hello Pi-Web, Sunday, January 27, 2008, 6:31:15 AM, you wrote: Hi Not sure what we have done - but snfilter has stoped working. The x.200801??.log.xml is not more created. SNFClient.exe.err says: /var/spool/snfilter/msg/20080127122626_4614.msg: Could Not Connect! Messages are put in /var/spool/snfilter/msg/ but not checked. I cant see what I have done wrong, but guess we did something around here: Jan 26 00:43 x37l67rv.20080125.log.xml as this is the last log. These are beeing created: -rw-r--r-- 1 root root 743591 Jan 27 12:29 x37l67rv.status.minute.20080127.log.xml -rw-r--r-- 1 root root 1079 Jan 27 12:30 x37l67rv.status.second.log.xml SNFServer has stopped --- if you had it set up as a service you should be able to restart it and solve the problem. If you were running it in a dos window -- start up a new dos window with it. Please look for any errors in your logs that might indicate why the SNFServer stopped. _M -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: GBUdb question
Hi Rob, You can add the IPs to GBUdbIgnoreList.txt if you want sniffer to ignore the IPs. Pete, I have some questions about GBUdb FIRST QUESTION: I have several clients who forward over e-mails from ISP accounts. I have a system whereby I can pick out the original sending server IP. I then add that IP to the message in a special header. (this can vary by ISP and situation, but I've programmed my system to appropriately determine which IP is the original sending server IP. Next, I add a special custom header which points out that IP. Would it be possible for MessageSniffer to grab the IP from a particular header (perhaps this header could be added as a node in the XML config file?). That way, if/when that header is available in the message, Sniffer would then treat *that* IP as the sender's IP? SECOND QUESTION: Is it possible to tell Sniffer to NOT allow the possibility of truncating on a message-by-message basis, where this would be determined if a special command line switch were present. In fact, can Sniffer be further instructed to ONLY run pattern matching scanning and ignore the GBUdb for that particular message? THIRD QUESTION: Much of the spam I block doesn't run through Sniffer. Additionally, many of the messages that Sniffer blocks are spams sent via established ISPs whereas I already have those IPs in an extensive whitelist that I've built up over the years. A 4% sampling of this whitelist can be found here: http://invaluement.com/fourpercentofwhitelist.txt (multiple the size of that by 25 to get an idea of the massive size of my IP whitelist) Here is what I'd like to do which I believe would make my contribution to sniffer most effective: (A) Have sniffer NOT automatically input data into GBUdb with each sniffer scan. (Is that possible?) (B) Alternatively, whenever my spam filter marks a message as spam, it will issue the following command (but ONLY if that IP is NOT on my IP whitelist, and regardless of whether or not the message was run through sniffer): SNFClient.exe -bad IP4Address (If on my IP whitelist, it just won't do anything here.) (C) If my spam filter marks a message as ham, then it will issue the following command (again, regardless of whether or not the message was run through sniffer) SNFClient.exe -good IP4Address ** ** I know that this puts more trust on me and my system, but I have also know that the quality of stats you'd receive from my system would vastly improved due to my abilities in this area and this would be a huge contribution to other Sniffer users over the norm. (I run one of the best RBLs and URI blacklists in the world... I know what I'm doing here!) Can these things be done? Rob McEwen # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Postfix
Hi We trying to setup snf with postfix. It seems to work - except it does not reject ant messages. The x.20080116.log.xml says: s u='20080116110805' m='20080116120805_22626.msg' code='69' error='ERROR_MSG_FILE'/ This I belive is because the msg file that is send to sniffer has a wrong format. - If true - how do we setup the right format for sniffer? # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Postfix
It seems right - but no go: In /var/spool/snfilter/msg/ -rw--- 1 snfilter snfilter 2965 Jan 16 18:35 20080116183528_10882.msg (deleted after process finished) Result: s u='20080116173528' m='20080116183528_10882.msg' code='69' error='ERROR_MSG_FILE'/ sniffer setup: SNIFFER_EXE=/var/spool/snfilter/SNFClient.exe AUTHENTICATION= INSPECT_DIR=/var/spool/snfilter/msg/ SENDMAIL=/usr/sbin/sendmail -i MSGFILE=`date +%Y%m%d%H%M%S`_$_$RANDOM.msg Hello Pi-Web, ERROR_MSG_FILE means that SNF could not open the file to be scanned. Be sure the you pass the full path of the message file and that permissions are correct so that SNF can open the file. Hope this helps, _M Wednesday, January 16, 2008, 12:31:58 PM, you wrote: No its not the message format. A message the get ERROR_MSG_FILE work fine on our windows SNF installation. Hi We trying to setup snf with postfix. It seems to work - except it does not reject ant messages. The x.20080116.log.xml says: s u='20080116110805' m='20080116120805_22626.msg' code='69' error='ERROR_MSG_FILE'/ This I belive is because the msg file that is send to sniffer has a wrong format. - If true - how do we setup the right format for sniffer? -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Postfix
Adding $INSPECT_DIR to the $SNIFFER_EXE $AUTHENTICATION $INSPECT_DIR$MSGFILE ||
{ command
Now it seems to work.
It seems right - but no go:
In /var/spool/snfilter/msg/
-rw--- 1 snfilter snfilter 2965 Jan 16 18:35 20080116183528_10882.msg
(deleted after process finished)
Result:
s u='20080116173528' m='20080116183528_10882.msg' code='69'
error='ERROR_MSG_FILE'/
sniffer setup:
SNIFFER_EXE=/var/spool/snfilter/SNFClient.exe
AUTHENTICATION=
INSPECT_DIR=/var/spool/snfilter/msg/
SENDMAIL=/usr/sbin/sendmail -i
MSGFILE=`date +%Y%m%d%H%M%S`_$_$RANDOM.msg
Hello Pi-Web,
ERROR_MSG_FILE means that SNF could not open the file to be scanned.
Be sure the you pass the full path of the message file and that
permissions are correct so that SNF can open the file.
Hope this helps,
_M
Wednesday, January 16, 2008, 12:31:58 PM, you wrote:
No its not the message format. A message the get ERROR_MSG_FILE work
fine on our windows SNF
installation.
Hi
We trying to setup snf with postfix.
It seems to work - except it does not reject ant messages.
The x.20080116.log.xml says:
s u='20080116110805' m='20080116120805_22626.msg' code='69'
error='ERROR_MSG_FILE'/
This I belive is because the msg file that is send to sniffer has a
wrong format.
- If true - how do we setup the right format for sniffer?
--
Mvh. Frank Jensen
[EMAIL PROTECTED]
www.pi.dk
Imponerende, fascinerende og kæmpe
Plakater f.eks. 149 x 149 = 629 kr
Vi kan også lave plakat fra dit digitale foto
www.plakatkunst.dk
#
This message is sent to you because you are subscribed to
the mailing list sniffer@sortmonster.com.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: One line nonsense mail
Is it possible to add own texts to SNF to include in the contents scan? Eg.: Subject: are unregulated and AND would be. by either the FSA or number of organisations. This way we could react at the first message recived. Hi All, I had like 37 different One line nonsense mail in my account today. (and so did our many of our users). Of cause they are not taken by SNF as almost all are different and from different IP sources. Is it a virus that generates such mails? Or what is the idea? Anyone having luck stopping these annoying mails? Basically the look like this: Subject: are unregulated and Body: would be. by either the FSA or number of organisations. Subject: Kitchen Body: God Rifle Leg Navy Subject: Post-office Body: Monster Spice Microscope Torch Subject: Room Body: Treadmill Shop Hammer Mouth # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Sniffer Win32 command line output
Make a bat fil like this: -- @echo off echo syntax batfilenavn.bat messagefil to test SNFclient.exe %1 echo %errorlevel% pause -- If it display zero the message is clean. Hello, I am evaluating Message Sniffer beta version but I am totally confused. :-) If I am in a MSDOS Window and I type: SNFClient.exe junkmsg.txt there is a very fast pause and I am returned to the command prompt. I can go into the log and see this: s u='20080110191039' m='junkmsg.txt' s='54' r='9649' m s='54' r='9649' i='383' e='391' f='m'/ p s='0' t='0' l='1577' d='39'/ /s So I know everything is working like it should be. But how do I get the result code for the spam message to output back to the command prompt? If I try to call SNFClient.exe from my C# code, I still cannot get a result code returned to me. I can get a result code if I do this: SNFClient.exe -test xx.xx.xx.xx but SNFClient.exe does not return the result code when I am passing a filename to be tested. Can someone point me in the right direction on how to see this result code via my C# software code or command prompt box? Thanks, Shawn -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] One line nonsense mail
Hi All, I had like 37 different One line nonsense mail in my account today. (and so did our many of our users). Of cause they are not taken by SNF as almost all are different and from different IP sources. Is it a virus that generates such mails? Or what is the idea? Anyone having luck stopping these annoying mails? Basically the look like this: Subject: are unregulated and Body: would be. by either the FSA or number of organisations. Subject: Kitchen Body: God Rifle Leg Navy Subject: Post-office Body: Monster Spice Microscope Torch Subject: Room Body: Treadmill Shop Hammer Mouth # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: I got a strong attack today
Hi I got a tool to test all messages in a folder with SNF. All with a non zero result is moved to a spam folder. Its like 84 lines of delphi code. If Pete will host the files I will supply the tool for free including source. Friday, January 4, 2008, 4:56:29 PM, you wrote: Hello I got a strong attack today, over thousand messages at the same time!! The usual technique: Impersonate the victim and send to non valid users of one domain of mine!! Changing IP for each message UNBELIEVABLE!! This is very common these days. We call it getting caught in the light. Our spamtrap server is currently experiencing a similar attack and is seeing 1850+ messages per minute. Luckily we've killed this particular campaign a few hours ago so leakage is only 7/min and 890+/min of these messages are being truncated (scan stopped based on IP via GBUdb) The only solution was, to stop all the services and move all the spool files in a temp directory. I won't use the nobody alias because at least the iMail Access Control can stop some bad IPs. My config is: Imail 9.23 Mxguard 3.1 Message Sniffer InvURIBL 3.7 Two questions: 1) There is a way or tool to recycle back good messages from the temp directory into the queue? You should be able to write a cmd script to test the messages in your temp folder against SNF and place the clean messages back into the spool for delivery. This doesn't give you a complete solution, but it is reasonably viable in such cases. I've not heard of it, but you may be able to find or write a similar utility to put the temp messages through the entire scan process at some reasonable pace -- You might ask DG about that - I'm not sure what would be the best way to go about that w/ mxGuard and he may have a solution already or know where it's buried. Side Note: We actually have a technology that we've simulated and not deployed called Gauntlet. Under certain conditions messages are shunted to a waiting area where their scanning and delivery are delayed for a period of time so that filtering systems can catch up... For example, messages that arrive from completely unknown IPs would have to run the gauntlet before being delivered. The sensitivity of the shunting system could be guided by storm data (B and C counts) from GBUdb to reduce the possibility of delaying ordinary messages. What you are describing is a manual version of this process. 2) How can I reduce or block(!) this kind of attacks? The new version of SNF is very good at reducing this kind of attack because the GBUdb component frequently can identify bad IP sources very quickly after a new campaign begins and is able to block many of the messages based on the IP reputation information known by the network. In some cases this might include substantially all of the attack prior to new pattern rules reaching your system -- in all cases at least some fraction of the attack would be identified (based on observations). The system will become more sensitive as more systems begin using the new software -- at this time it is remarkably sensitive even though only a small fraction of SNF users are already using it -- so we expect significant improvements. In this case, for example, many of the messages arriving would be seen by SNF, identified after a very short scan (only the first few hundred bytes), and then most-likely deleted (depending on how you tune your system; also I'm not sure what options are available from mxGuard w/ regard to preempting additional tests and/or test ordering). Given your system's configuration I don't know of any way to block this kind of attack without adding additional components. A couple that come to mind are SPF checking (so that any message pretending to come from your domains must actually be coming from your servers before being accepted), and graylisting which, while sometimes problematic, currently provides some pretty good protection against dumb-bot attacks. (Note that the newer bot softwares out there easily defy gray listing so it's effectiveness is dropping quickly) Hope this helps, Best, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED]
[sniffer] Re: Excessive amounts of spam
We have been running it for - I guess - 2 month now without any trouble. How stable is the beta version? Regards David Moore [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] J.P. MCP, MCSE, MCSE + INTERNET, CNE. www.adsldirect.com.au http://www.adsldirect.com.au/ for ADSL and Internet www.romtech.com.au http://www.romtech.com.au/ for PC sales Office Phone: (+612) 9453 1990 Fax Phone: (+612) 9453 1880 Mobile Phone: +614 18 282 648 Skype Phone: ADSLDIRECT POSTAL ADDRESS: PO BOX 190 BELROSE NSW 2085 AUSTRALIA. - This email message is only intended for the addressee(s) and contains information that may be confidential, legally privileged and/or copyright. If you are not the intended recipient please notify the sender by reply email and immediately delete this email. Use, disclosure or reproduction of this email, or taking any action in reliance on its contents by anyone other than the intended recipient(s) is strictly prohibited. No representation is made that this email or any attachments are free of viruses. Virus scanning is recommended and is the responsibility of the recipient. - *From:* Message Sniffer Community [mailto:[EMAIL PROTECTED] *On Behalf Of *Pete McNeil *Sent:* Friday, 21 December 2007 8:10 AM *To:* Message Sniffer Community *Subject:* [sniffer] Re: Excessive amounts of spam Hello David, Thursday, December 20, 2007, 3:25:45 PM, you wrote: Ø If you are not yet running the latest beta then that might help quite a bit since the GBUdb (IP reputation system) does a good job capturing new spam from old bots even before rules are coded. Please clarify are you saying it would help if we had the beta installed? Yes. The new GBUdb engine reduces leakage quite a bit. As more systems adopt the new version this will improve even more. Most new spam campaigns are started with some large fraction of existing bots. Messages from bots that have already been identified will be blocked even before new content rules can be generated (if needed). _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Excessive amounts of spam
We are using sniffer and free tools: yasu (URLBL) and RBLCHECK (DNSBL). URLBL does catch some that sniffer dont. URLBL I think has as low false rate as sniffer - but it does not catch as many as sniffer. DNSBL also (mainly spamcop), but with much more false than sniffer. We have added a IP whitelist for DNSBL to lower the false rate. We used to run spam assassin, but the above config has much lower false and uses much less cpu. Frank: Thanks for your input. There are definitely things leaking though that wouldn't have leaked through before. We've held off hoping for a production release but it may not be practical much longer. On that note, for anyone else in the same position, we tested adding InvURIBL from Invariant Systems. It's not a sniffer replacement but definitely caught a lot of what sniffer currently lets through for the very valid reasons Pete has covered. The only thing missing seemed to be a white list so that you could white list legitimate publications that might contain links to 'offensive' sites. That can probably be tuned out thru weighting however we'd hoped not to be re-inventing the wheel for a short term solution. Eric - Original Message - From: Pi-Web - Frank Jensen [EMAIL PROTECTED] To: Message Sniffer Community sniffer@sortmonster.com Sent: Thursday, December 20, 2007 1:17 PM Subject: [sniffer] Re: Excessive amounts of spam We have been running it for - I guess - 2 month now without any trouble. How stable is the beta version? Regards David Moore [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] J.P. MCP, MCSE, MCSE + INTERNET, CNE. www.adsldirect.com.au http://www.adsldirect.com.au/ for ADSL and Internet www.romtech.com.au http://www.romtech.com.au/ for PC sales Office Phone: (+612) 9453 1990 Fax Phone: (+612) 9453 1880 Mobile Phone: +614 18 282 648 Skype Phone: ADSLDIRECT POSTAL ADDRESS: PO BOX 190 BELROSE NSW 2085 AUSTRALIA. - This email message is only intended for the addressee(s) and contains information that may be confidential, legally privileged and/or copyright. If you are not the intended recipient please notify the sender by reply email and immediately delete this email. Use, disclosure or reproduction of this email, or taking any action in reliance on its contents by anyone other than the intended recipient(s) is strictly prohibited. No representation is made that this email or any attachments are free of viruses. Virus scanning is recommended and is the responsibility of the recipient. - *From:* Message Sniffer Community [mailto:[EMAIL PROTECTED] *On Behalf Of *Pete McNeil *Sent:* Friday, 21 December 2007 8:10 AM *To:* Message Sniffer Community *Subject:* [sniffer] Re: Excessive amounts of spam Hello David, Thursday, December 20, 2007, 3:25:45 PM, you wrote: Ø If you are not yet running the latest beta then that might help quite a bit since the GBUdb (IP reputation system) does a good job capturing new spam from old bots even before rules are coded. Please clarify are you saying it would help if we had the beta installed? Yes. The new GBUdb engine reduces leakage quite a bit. As more systems adopt the new version this will improve even more. Most new spam campaigns are started with some large fraction of existing bots. Messages from bots that have already been identified will be blocked even before new content rules can be generated (if needed). _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative
[sniffer] Re: XCI Error!: snf_EngineHandler::MaxEvals
The SNFserver.exe is present on the task list, so it will not automatic restart. ERROR in todays log: e u='20071102100405' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102100539' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102100714' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102100835' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102100956' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102113453' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102114259' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102114429' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102114601' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102114726' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102114836' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ s u='20071102131619' m='C:\Program Files\Merak\temp\200711021416181623.tmp' code='72' error='ERROR_MAX_EVALS'/ e u='20071102133320' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ s u='20071102174238' m='c:\...tmp' code='69' error='ERROR_MSG_FILE'/ ... cut a lot... s u='20071102174358' m='c:\...tmp' code='69' error='ERROR_MSG_FILE'/ e u='20071102204218' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ p before/after max_evals: - What does that tell me? p s='0' t='411' l='34867' d='72'/ ... s u='20071102131619' m='C:\Program Files\Merak\temp\200711021416181623.tmp' code='72' error='ERROR_MAX_EVALS'/ ... p s='0' t='80' l='4140' d='93'/ Hello Pi-Web, Friday, November 2, 2007, 2:01:30 PM, you wrote: snip/ 31st oct. spam level raised, SNF was not validating the mails, the snfclient.exe.err shows lines like: C:\Program Files\Merak\temp\2007110215013101AF.tmp: Could Not Connect! Could not connect indicates (most likely) that the SNFServer was down. Any time the client produces a .err it is unusual. Normal errors are reported to the SNFengine's log file(s). We restrated the SNFserver (running as a service) and scans run smoothly until today (2nd nov.), where same issue happen: Could Not Connect!. No errors in between. Something is knocking the server offline. The log also show: (first line). C:\Program Files\Merak\temp\200711021416181623.tmp: XCI Error!: snf_EngineHandler::MaxEvals Think this MaxEvals is what cause the error. Is it due to the engine getting to many mails to evaluate? No. MaxEvals is a condition that is theoretically possible but extremely rare. As a message is scanned, little creatures called evaluators are created and re-used during the scan to identify any patterns that might exist in the message. The scan depth metric indicates the peak number of evaluators that were alive during the scan. Normally this number is between 60 and 150 though it changes all the time. In order to detect possible rulebase corruption there is a hard-coded limit to the number of evaluators that are allowed to live for a particular scan. It is possible that this number needs to be adjusted. That hasn't happened in a while - but since you're not getting any other errors (that we know of) that's the most likely scenario. The number of evaluators that are alive at one time for a particular scan depends on the active rules in the rulebase and the data in the message. The number is almost impossible to predict though it does (and should) normally stay in a fairly restricted range. How do we avoid this? First, let's verify that there were no other errors. Please look in your snf log files and check for any e/ elements. These will describe any other errors that occurred. If we find no other errors then I will make an adjustment to the maximum evals metric and we will go from there. While you are in your logs -- look a the p/ (performance) elements and get an idea what the scan depth is typically. That will help us compare your system to others and to determine what the new limit should be. Originally the scan depth limit was designed to help detect possible corruption or unexpected conditions in the scanning engine. It's been there since the first version. It's a kind of sanity check -- Most likely it just needs to be adjusted since spam has changed so much over the years. In the early days scan depths were consistently well below 100 -- even in the 40-60 range. These days there are more abstracts in the rulebase so more creatures are required to get a comprehensive idea of what is in each message. Another thing I will look at is that this exception should be handled gracefully. I will look into this -- it may be that we want the SNFServer to fail under these conditions because it is a clue to something being out of adjustment -- In this case, probably just the limit setting. In the mean time, if you automatically restart your SNFServer after a failure it should be safe and will pick up any waiting
[sniffer] Re: XCI Error!: snf_EngineHandler::MaxEvals
On 8438 t today we got a average T=111,1176819 Min=0, Max=7211. (57 scans took above 1000, 6384 scans took less than 101). The server is rather old and serving both web mail, pop3 and smtp. And heavy usage of web mail does slow it down. This might be the case on the slow scans. The long scans is not at the same time, but from time to time during the day. Still this should not lock up snfserver. To call snf we use a dll of own development (pluged in to Merak mail server). The call to snfclient is done using a: WaitforSingleObject with INFINITE wait time. (perhaps we should change this). When it finish - and it does - we get the snf result using GetExitCodeProcess. This return zero (whitch is good, else all messages would be rejected) when the snfserver is in the Could Not Connect! state. Friday, November 2, 2007, 5:04:47 PM, you wrote: The SNFserver.exe is present on the task list, so it will not automatic restart. ERROR in todays log: snip/ e u='20071102100835' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102100956' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ e u='20071102113453' context='SNF_NETWORK' code='99' text='ERROR_SYNC_FAILED'/ snip/ The ERROR_SYNC_FAILED errors are caused by network congestion between your systems and ours. Ping times are well above 120ms at the moment, for example. I note that there are periods of time when there is no trouble making the connection and your current telemetry also looks good so we can ignore that error for the time being. Your latest SYNC took only 290ms and occurred with no retries. Here is my telemetry on that: session-data time=290 standby=15 cycles=6 sent=424 received=1930 comms=Completed Ok success=true s u='20071102131619' m='C:\Program Files\Merak\temp\200711021416181623.tmp' code='72' error='ERROR_MAX_EVALS'/ The above scan s/ failed due to too many evaluators. s u='20071102174238' m='c:\...tmp' code='69' error='ERROR_MSG_FILE'/ ... cut a lot... s u='20071102174358' m='c:\...tmp' code='69' error='ERROR_MSG_FILE'/ ERROR_MSG_FILE indicates that the SNFServer program was unable to open or read the file. Something must have removed it before it could be processed. This error is unrelated to the SYNC and MAX_EVALS errors. I also noted that the SYNC errors do not seem to coincide closely with the MSG_FILE errors. For now we will need to treat all three as separate cases. On some systems we have found cases where the system becomes so busy that scans take too long and are then cancelled before they are complete. This condition might account for some of the MSG_FILE errors. Is there a timeout on the mechanism that calls the SNFClient? If there is, then we might be able to mitigate the ERROR_MSG_FILE condition by extending that timeout. Considering the SYNC errors -- they are not critical because the SNF engine will tolerate them provided it is able to make a connection most of the time. When a connection is made and the SYNC session is successful then all of the data from previously unsuccessful sessions is transferred in the process. p before/after max_evals: - What does that tell me? p s='0' t='411' l='34867' d='72'/ The p/ element always belongs to an s/ element. An s/ element represents a single message scan. The p/ element describes the system's performance during that scan. In the case of the p/ element above, it took 0ms to setup the scan (read the file etc) and then took 411ms to perform the scan. This would usually indicate that your system is CPU bound. Normally an SNF scan will take a very short time. This one took almost half a second. The l indicates the length of the message scan in bytes and the d indicates the scan depth. That is, the maximum number of evaluators that were alive during the scan. ... s u='20071102131619' m='C:\Program Files\Merak\temp\200711021416181623.tmp' code='72' error='ERROR_MAX_EVALS'/ ... p s='0' t='80' l='4140' d='93'/ The p/ element here does not belong to the s/ element. It belongs to a different scan. Once the s/ element closes (with /s) anything after that point belongs to a different event. --- I don't have any other reports of MAX_EVAL errors. That doesn't mean that they are not out there, but it does mean that they are not usually a problem for other folks. I'm not sure what can be causing your SNFServer to crash -- it should not be MAX_EVAL errors. They are handled safely by the code according to what I've seen so far in my search. None the less, I will be increasing the max eval setting in the next release and I will push it out sooner rather than later. Since you have reported this problem I won't wait for the other features before pushing out beta 1.6. If I can get to it tonight I will. In the mean time, do you have any idea what might be causing your CPU to be so heavily loaded that your SNF scans are taking 400+ milliseconds? Do you have many p/ records that show high t values like that? (I do see the
[sniffer] Re: Bad Rule: 1604021
Hi Pete, We have fileret out 169 mails based on this rule. Most are spam. I have just collected the latest rulebase - it is from 20.00, The false positive are still taken as spam. If you want the 169 please let me know. --22:37:49-- http://www.sortmonster.net/Sniffer/Updates/xx.snf = `xx.new.gz' Resolving www.sortmonster.net... done. Connecting to www.sortmonster.net[207.97.242.65]:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [application/x-sortmonster] Server file no newer than local file `xxx.snf' -- not retrieving. Hello Sniffer Folks, This is an alert about a potentially bad rule 1604021. The rule was an abstract pattern for some of today's image spam. Indications are that the final coding was too broad. The rule was in place for approximately 5 hours ending about 30 minutes ago. Some differences in timing are inevitable since all rulebases are compiled individually. If you have the ability to release and rescan from quarantine based on SNF rule IDs then we recommend executing that process against this rule id: 1604021. Hope this helps, Thanks, _M -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Bad Rule: 1604021
Hi Pete, Checked all manuelly, 7 of 155 was good. The new rule database don't match on any of the 7. All 155 is matched as spam. Monday, October 15, 2007, 4:43:03 PM, you wrote: Hi Pete, We have fileret out 169 mails based on this rule. Most are spam. That's good to hear. Thanks! _M -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: False Positives
Hi Pete, Actually it is true ;-) http://kb.armresearch.com/index.php?title=Message_Sniffer.FAQ.FalsePositives#What_are_the_guidelines_for_sending_a_False_Positive.3F - Please include your license ID in your message and send the messages from your registered email address. Email from non-registered email addresses and email having no license ID may be ignored. It should be impossible to guess such a standard rule it is still unique to each system. We are also having trouble with your response to our false. Hello Pi-Web, Actually that's not true - the license ID is normally derived from information in the headers as the message is processed by special software on our system. This is a very rare case - and where any kind of white rule is concerned we like to keep each case specific and extremely difficult to guess. Any kind of standard white rule is a problem waiting to happen. Hope this helps, _M Monday, October 8, 2007, 11:45:46 AM, you wrote: Why not add the license code as local whitelist string in each database, the license code is normaly supplied in the false report mail anyway. Is there any way of getting false positives to you other than emailing them to [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ? My dilemma is that I'm using the same SMTP server for sending email out as getting email in -- so outgoing email is getting spam checked using Message Sniffer -- This means my False Positive reports are getting quarantined before they can get out!! Any ideas on how to work around this problem? We should be able to create a local white rule for this purpose. Or, you could create a local white-list entry of some kind in your other filtering systems (those that call SNF). Send me a note off-list from the address system that you will use to submit false positives and include information on any other systems you will use to submit false positives to us. We will work through it to create an appropriate white rule for this purpose. Hope this helps, Thanks, _M -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: False Positives
Why not add the license code as local whitelist string in each database, the license code is normaly supplied in the false report mail anyway. Is there any way of getting false positives to you other than emailing them to [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ? My dilemma is that I'm using the same SMTP server for sending email out as getting email in -- so outgoing email is getting spam checked using Message Sniffer -- This means my False Positive reports are getting quarantined before they can get out!! Any ideas on how to work around this problem? We should be able to create a local white rule for this purpose. Or, you could create a local white-list entry of some kind in your other filtering systems (those that call SNF). Send me a note off-list from the address system that you will use to submit false positives and include information on any other systems you will use to submit false positives to us. We will work through it to create an appropriate white rule for this purpose. Hope this helps, Thanks, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: False Positive - how to react?
For the first (known) time I see Message Sniffer filter a valid mail. What is the best way to handle stuff like this? Check out this page: http://kb.armresearch.com/index.php?title=Message_Sniffer.FAQ.FalsePositives # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: False Positive - how to react?
Ok, I guess the instruction is for people who filter spam to a spam folder... ;-) I think you should contact [EMAIL PROTECTED] I think they will able to remove the rule based on you sniffer log, perhaps the only will remove it for your system. One other problem - the first entry og the log is your licensecode! - you should not post it public (e.g. in this group). This is the related Sniffer log entry: *** 20070926071222 d064801a658d9.smd 0 78 Match 1336961 60 6933694583 *** 20070926071222 d064801a658d9.smd 0 78 Final 1336961 60 0 26005 83 thanks for your answer. Seems like i'm too dense to get it. The step by step instructions tell me: 4. Attach the message that was captured incorrectly. How should I do that? I don't have that message because it got filtered and deleted by Sniffer. For the first (known) time I see Message Sniffer filter a valid mail. What is the best way to handle stuff like this? Check out this page: http://kb.armresearch.com/index.php?title=Message_Sniffer.FAQ.FalsePositives -- Mvh. Frank Jensen [EMAIL PROTECTED] www.pi.dk Imponerende, fascinerende og kæmpe Plakater f.eks. 149 x 149 = 629 kr Vi kan også lave plakat fra dit digitale foto www.plakatkunst.dk # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
