[sniffer] Re: My rulebase download and log upload script
Weekend, what is that? Thanks Andrew. John T eServices For You "Seek, and ye shall find!" > -Original Message- > From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of > Colbeck, Andrew > Sent: Friday, July 07, 2006 6:24 PM > To: Message Sniffer Community > Subject: [sniffer] My rulebase download and log upload script > > The last thing before I leave for the weekend... > > I finally got around to updating my download/upload script so that I can > upload compressed logs. > > In the course of doing that, I found that my upgraded version of wget > has changed its behaviour; as of the 1.10.x series, if you specify -O to > specify the target filename, various options are ignored including the > -N for "download only if server side is newer". Therefore, ever since I > upgraded my wget, I've been downloading a compressed rulebase file on > *each* run. > > Some of this script is antique and some of it is new. I just downloaded > the "standard download script" that Bill Landry ushered into this world, > and my script was certainly informed by the discussions of that on this > list. > > (I'm not trying to replace that script, I'm just giving credit where > credit is due.) > > My .cmd file script is attached as a .txt file; as I mentioned a while > back, I use both the IMail "external script" mailbox method to launch > this file when SortMonster/ARM sends me my notification, and I also run > it on a schedule with the AT command so that one of them will work to > get timely updates. > > Andrew 8) > # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
[sniffer] My rulebase download and log upload script
The last thing before I leave for the weekend... I finally got around to updating my download/upload script so that I can upload compressed logs. In the course of doing that, I found that my upgraded version of wget has changed its behaviour; as of the 1.10.x series, if you specify -O to specify the target filename, various options are ignored including the -N for "download only if server side is newer". Therefore, ever since I upgraded my wget, I've been downloading a compressed rulebase file on *each* run. Some of this script is antique and some of it is new. I just downloaded the "standard download script" that Bill Landry ushered into this world, and my script was certainly informed by the discussions of that on this list. (I'm not trying to replace that script, I'm just giving credit where credit is due.) My .cmd file script is attached as a .txt file; as I mentioned a while back, I use both the IMail "external script" mailbox method to launch this file when SortMonster/ARM sends me my notification, and I also run it on a schedule with the AT command so that one of them will work to get timely updates. Andrew 8) echo off REM This file is C:\Utils\UpdateSniffer.cmd REM AC July-06-2003 REM The purpose is to truncate the MessageSniffer log and fetch a fresh copy of the REM spam signatures rulebase. REM Dependencies: REM wget.exe package (free) from hhttp://xoomer.alice.it/hherold/ (not http://www.gnu.org/software/wget/wget.html which lags!) REM gunzip.exe (free) from http://www.gzip.org/ REM fgrep.exe (free) from http://www.gnu.org/software/grep/grep.html REM timeout.exe (free) part of Microsoft Windows Server Resource Kit REM UploadScript.txt this is the text of our side of the ftp upload conversation, now built on the fly REM REM Folders used: REM d:\MessageSniffer REM d:\MessageSniffer\Download REM d:\MessageSniffer\OldLogs REM REM Oct-31-2004 AC updated to incorporate .gz compression when a download is made REMa big thanks to Bill Landry for a working example! REM Oct-30-2004 AC updated to skip the download with wget -N if it is a duplicate of a previous download REMand also to skip the update if the download was a duplicate REM Sep-07-2004 AC updated to make use of the sniffer check executable REM Jun-07-2005 AC renamed mailsniffer folder to MessageSniffer REM Jul-07-2006 AC wget defaults have changed, the (newer) option no longer works with the (rename download) option REMand I've added extra safety checking logic and gzipping, and on-the-fly build of the REMFTP upload script. REM LinceseID=This is the 8 character text that defines your rulebase and executable name. set LicenseID=12345678 REM Authorization=This is the 16 character text that is your "password" and you must keep secret. set Authorization=1234567890123456 REM FQDN=This is the fully qualified domain name at which you receive your email, e.g. bentall.com set FQDN=example.com set SnifferDrive=d: set SnifferFolder=d:\MessageSniffer if "%FQDN%" == "example.com" (echo No, really. You must set the FQDN variable to your own domain name. Now quitting. & goto quit) REM Change the current directory to the Message Sniffer drive and directory %SnifferDrive% cd %SnifferFolder% echo Clearing the old result files and logs if exist wgetresult.txt del wgetresult.txt REM Use the -N parameter to only download a newer file; note that as of wget v1.1x this is no longer REM compatible with the -O parameter to specify the destination filename. REM Note the use of the gzip encoding to save bandwidth. REM We use the -o parameter and create a new logfile each time so that we can grep the result if not exist %SnifferFolder%\Download md %SnifferFolder%\Download wget --tries=10 --wait=5 --random-wait --directory-prefix=%SnifferFolder%\Download -o wgetresult.txt --header=Accept-Encoding:gzip -N --http-user=sniffer --http-passwd=ki11sp8m http://www.sortmonster.net/Sniffer/Updates/%LicenseID%.snf REM If we didn't get the file, at least rotate the log file before quitting if errorlevel 1 goto logrot REM The errorlevel is zero whether we downloaded a fresh copy or skipped the download. REM Let's find out if we can skip the checking and backup and go straight to the log rotation and upload. REM If this grep is succesfull, the errorlevel is zero. fgrep "Server file no newer than local file" wgetresult.txt if not errorlevel 1 goto logrot ren %SnifferFolder%\Download\%LicenseID%.snf %LicenseID%.snf.gz copy %SnifferFolder%\Download\%LicenseID%.snf.gz %SnifferFolder%\Download\%LicenseID%.snf.gz.bak REM Unzip the compressed file; gunzip will automatically hack off the extension, REM creating %LicenseID%.snf and it will delete the .gz file too echo Now testing the wget'ed file gunzip -d -f -v %SnifferFolder%\Download\%LicenseID%.snf.gz REM However, sometimes wget downloads a zero length file, in which case the gunzip fails if errorlevel 1 (echo Some k
[sniffer] Re: Lot of stock spam getting through....
Hello Darin, Thanks everyone, kind words are much appreciated. I must share them with the rest of the SNF team who also work 24x7 to make this happen. You don't see them often but I couldn't do it without them. This seems like a good time to introduce a few of them and thank them publicly for their efforts (We've come a long way in a year!): Linda (TechGirl) [Accounting/Ops], Karen (Tink) [Ops/Support/Web], Jason (the Bag) [Filter Team Leader], Adam (TheFelcher) [Rule-Tech], Baron (Kojak) [Rule-Tech], Michael U [Rule-Tech], Nick G [Rule-Tech], Michael M [Exec], Scott C [Sales/Marketing], Joel S [Hosting/Sourcing], ...then there's me... Pete (Madscientist) [Science/Development], Plus a bunch of folks (too many to list everyone) who help out from time to time in too many ways to count. Ok... work to do... _M Friday, July 7, 2006, 1:12:50 PM, you wrote: > Great job, Pete! And thanks for all of your efforts to simultaneously > increase the catch rate and decrease the FP rate. > Darin. -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
[sniffer] Re: Lot of stock spam getting through....
Great job, Pete! And thanks for all of your efforts to simultaneously increase the catch rate and decrease the FP rate. Darin. - Original Message - From: "Pete McNeil" <[EMAIL PROTECTED]> To: "Message Sniffer Community" Sent: Friday, July 07, 2006 11:11 AM Subject: [sniffer] Re: Lot of stock spam getting through Hello Chuck, Friday, July 7, 2006, 10:48:28 AM, you wrote: > We are seeing a lot of stock spam that is only a picture image getting > through sniffer. I had a big fight with one like that all last night -- there are some unusual characters in the message that made it hard to filter and it took some time to do the analysis (picking through them with a hex editor). I think these are handled now (as of about 0400e this morning) as I don't have any getting through spamtraps at the moment. I will look into it again. _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]> # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
[sniffer] Re: Lot of stock spam getting through....
Hello George, Thanks very much! _M Friday, July 7, 2006, 11:18:24 AM, you wrote: > Hi Pete, > > I've been a customer for a couple of years and usually don't have > much to say via maillists. But I wanted to take a moment this > morning and think you for the work you do. Keeping up with this > stuff must force you to keep your nose to the grindstone. I really > appreciate your work. > Thanks again, > > George Thompson > Cheif Technical Officer > Levelfield.com, Inc > www.levelfield.com > DBA OnlineAgency.com > www.onlineagency.com > building the Internet one small business at a time > >> I had a big fight with one like that all last night -- there are some >> unusual characters in the message that made it hard to filter and it >> took some time to do the analysis (picking through them with a hex >> editor). >> >> I think these are handled now (as of about 0400e this morning) as I >> don't have any getting through spamtraps at the moment. I will look >> into it again. >> >> _M >> >> -- >> Pete McNeil >> Chief Scientist, >> Arm Research Labs, LLC. > -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
[sniffer] Re: Lot of stock spam getting through....
I agree! I could tell Pete was burning the midnight oil by the pattern of Sniffer Updates this morning. Thanks Pete! Bill Green dfn Systems 505-622-7853 [EMAIL PROTECTED] - Original Message - From: George Thompson To: Message Sniffer Community Sent: Friday, July 07, 2006 9:18 AM Subject: [sniffer] Re: Lot of stock spam getting through Hi Pete,I've been a customer for a couple of years and usually don't have much to say via maillists. But I wanted to take a moment this morning and think you for the work you do. Keeping up with this stuff must force you to keep your nose to the grindstone. I really appreciate your work.Thanks again,George ThompsonCheif Technical OfficerLevelfield.com, Incwww.levelfield.comDBA OnlineAgency.comwww.onlineagency.combuilding the Internet one small business at a time> I had a big fight with one like that all last night -- there are some> unusual characters in the message that made it hard to filter and it> took some time to do the analysis (picking through them with a hex> editor).> > I think these are handled now (as of about 0400e this morning) as I> don't have any getting through spamtraps at the moment. I will look> into it again.> > _M> > --> Pete McNeil> Chief Scientist,> Arm Research Labs, LLC.
[sniffer] Re: Lot of stock spam getting through....
Hi Pete, I've been a customer for a couple of years and usually don't have much to say via maillists. But I wanted to take a moment this morning and think you for the work you do. Keeping up with this stuff must force you to keep your nose to the grindstone. I really appreciate your work. Thanks again, George Thompson Cheif Technical Officer Levelfield.com, Inc www.levelfield.com DBA OnlineAgency.com www.onlineagency.com building the Internet one small business at a time > I had a big fight with one like that all last night -- there are some > unusual characters in the message that made it hard to filter and it > took some time to do the analysis (picking through them with a hex > editor). > > I think these are handled now (as of about 0400e this morning) as I > don't have any getting through spamtraps at the moment. I will look > into it again. > > _M > > -- > Pete McNeil > Chief Scientist, > Arm Research Labs, LLC.
[sniffer] Re: Lot of stock spam getting through....
Hello Chuck, Friday, July 7, 2006, 10:48:28 AM, you wrote: > We are seeing a lot of stock spam that is only a picture image getting > through sniffer. I had a big fight with one like that all last night -- there are some unusual characters in the message that made it hard to filter and it took some time to do the analysis (picking through them with a hex editor). I think these are handled now (as of about 0400e this morning) as I don't have any getting through spamtraps at the moment. I will look into it again. _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
[sniffer] Re: Lot of stock spam getting through....
We are getting the same, Tons of the, also a bunch of blank emails. Cordially, Heimir Eidskrem i360, Inc. 2825 Wilcrest, Suite 675 Houston, TX 77042 Ph: 713-981-4900 Fax: 832-242-6632 [EMAIL PROTECTED] www.i360.net www.i360hosting.com www.realister.com Houston's Leading Internet Consulting Company Chuck Schick wrote: We are seeing a lot of stock spam that is only a picture image getting through sniffer. Chuck Schick Warp 8, Inc. (303)-421-5140 www.warp8.com # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]> # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
[sniffer] Lot of stock spam getting through....
We are seeing a lot of stock spam that is only a picture image getting through sniffer. Chuck Schick Warp 8, Inc. (303)-421-5140 www.warp8.com # This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
