RE: [Declude.JunkMail] Log analysis and test check scripts

2004-04-22 Thread Paul Fuhrmeister
Thank you Bill and Roger for sharing your excellent work.  

[EMAIL PROTECTED]


 The scripts run under both Windows NT 4 and Windows 2000. They are 
 pure Windows command scripts and therefore not as fast as some of the 
 other log analysis tools. The analyses below took about one minute 
 each in all mode.

Took a bit longer on my system but there were 230,000 messages.  In
comparing the results with my program (WAMLOG) they were within 0.2%!  

Your program:

WEIGHT10 218863
WEIGHTdel 207491 

My Program:

WEIGHT10 218866
WEIGHTDEL 207493

I didn't know command script was so powerful.  Only about 100 lines of code!
I wrote my program in C++ and it took about 300 lines of code :)

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] Log analysis and test check scripts

2004-04-21 Thread Nick Hayer
On 21 Apr 2004 at 21:24, Roger Eriksson wrote:

*very* nice job Roger - 

Thanks!

-Nick Hayer

 Hi,
 
 My log analysis and test check scripts are available for download at:
 http://www.botany.gu.se/download/decludescript/LOG_analysis.zip
 http://www.botany.gu.se/download/decludescript/TEST_check.zip
 
 The first script creates a list with the number of hits for each test,
 number of messages that are OK or whitelisted, and a spam summary
 (incoming messages, deleted spam, held spam, marked spam, non-spam).
 
 The second script does a pairwise comparison between a specific test
 and all other tests regarding number of individual hits and number of
 shared hits (i.e. messages that fail both tests).
 
 Both scripts have two modes, one where the analysis is based on all
 message hits and another where it is based on unique messages only
 (i.e. a message hit is only counted once irrespective of the number of
 recipients). The first mode is much faster, but they can give some
 interesting results when compared.
 
 The scripts run under both Windows NT 4 and Windows 2000. They are
 pure Windows command scripts and therefore not as fast as some of the
 other log analysis tools. The analyses below took about one minute
 each in all mode.
 
 /Roger
 
 
 == Output from the log analysis script ==
 
 Declude test results -- dec0420.log
 
 --- Total number of hits --
 
 AHBL-PROXY 4197
 AHBL-RHSBL 1296
 AHBL-SOURCE 362
 BADHEADERS 2523
 BASE64-PLUS 381
 BASE64 762
 CBL 16295
 COMMENTS 64
 DSBL 14287
 DSN 2837
 FORGEDLOCAL 685
 GREYLIST 6
 HELOBOGUS 5812
 MAILFROM 1233
 MAILPOLICE 902
 MESSAGE OK 2672
 NETBL 563
 OPM 1945
 ORDB 48
 REVDNS 5752
 RSL 1815
 SBL 877
 SNIFFER-ADULT 2860
 SNIFFER-CASINO 44
 SNIFFER-CREDIT 685
 SNIFFER-EMAIL 87
 SNIFFER-EXP 1494
 SNIFFER-GEN 1374
 SNIFFER-GREY 5
 SNIFFER-INSUR 661
 SNIFFER-MAL 2
 SNIFFER-MEDIA 2437
 SNIFFER-OBFUSC 555
 SNIFFER-PHARM 5964
 SNIFFER-PRINT 10
 SNIFFER-RICH 889
 SNIFFER-SCAM 107
 SNIFFER-TOOLS 1
 SNIFFER-TRAVEL 19
 SNIFFER 17194
 SORBS-DUHL 10199
 SPAMCOP 17652
 SPAMDOMAINS 3895
 SPAMHEADERS 184
 SPAMTRAP 150
 SPFFAIL 405
 SURBL 2761
 URLDBL 152
 WEIGHT15-19 553
 WEIGHT20 18482
 WHITELISTED 530
 
 - Total number of messages 
 
 Incoming: 21154
 Held spam: 18482 (87%)
 Marked spam: 553 (2%)
 Non-spam: 2119 (10%)
 
 
 == Output from the test check script ==
 
 Test check results -- dec0420.log
 
 ---
 Test: SBL
 Total number of hits: 877
 ---
 Shared with AHBL-PROXY (4197 hits): 58 (6%)
 Shared with AHBL-RHSBL (1296 hits): 137 (15%)
 Shared with AHBL-SOURCE (362 hits): 314 (35%)
 Shared with BADHEADERS (2523 hits): 172 (19%)
 Shared with BASE64-PLUS (381 hits): 13 (1%)
 Shared with BASE64 (762 hits): 15 (1%)
 Shared with CBL (16295 hits): 355 (40%)
 Shared with COMMENTS (64 hits): 6 (0%)
 Shared with DSBL (14287 hits): 165 (18%)
 Shared with DSN (2837 hits): 94 (10%)
 Shared with FORGEDLOCAL (685 hits): 23 (2%)
 Shared with GREYLIST (6 hits): 0 (0%)
 Shared with HELOBOGUS (5812 hits): 317 (36%)
 Shared with MAILFROM (1233 hits): 21 (2%)
 Shared with MAILPOLICE (902 hits): 371 (42%)
 Shared with NETBL (563 hits): 15 (1%)
 Shared with OPM (1945 hits): 2 (0%)
 Shared with ORDB (48 hits): 0 (0%)
 Shared with REVDNS (5752 hits): 445 (50%)
 Shared with RSL (1815 hits): 2 (0%)
 Shared with SNIFFER-ADULT (2860 hits): 219 (24%)
 Shared with SNIFFER-CASINO (44 hits): 7 (0%)
 Shared with SNIFFER-CREDIT (685 hits): 99 (11%)
 Shared with SNIFFER-EMAIL (87 hits): 82 (9%)
 Shared with SNIFFER-EXP (1494 hits): 77 (8%)
 Shared with SNIFFER-GEN (1374 hits): 33 (3%)
 Shared with SNIFFER-GREY (5 hits): 0 (0%)
 Shared with SNIFFER-INSUR (661 hits): 39 (4%)
 Shared with SNIFFER-MAL (2 hits): 0 (0%)
 Shared with SNIFFER-MEDIA (2437 hits): 32 (3%)
 Shared with SNIFFER-OBFUSC (555 hits): 30 (3%)
 Shared with SNIFFER-PHARM (5964 hits): 156 (17%)
 Shared with SNIFFER-PRINT (10 hits): 9 (1%)
 Shared with SNIFFER-RICH (889 hits): 84 (9%)
 Shared with SNIFFER-SCAM (107 hits): 1 (0%)
 Shared with SNIFFER-TOOLS (1 hits): 1 (0%)
 Shared with SNIFFER-TRAVEL (19 hits): 2 (0%)
 Shared with SNIFFER (17194 hits): 871 (99%)
 Shared with SORBS-DUHL (10199 hits): 197 (22%)
 Shared with SPAMCOP (17652 hits): 659 (75%)
 Shared with SPAMDOMAINS (3895 hits): 94 (10%)
 Shared with SPAMHEADERS (184 hits): 34 (3%)
 Shared with SPAMTRAP (150 hits): 0 (0%)
 Shared with SPFFAIL (405 hits): 0 (0%)
 Shared with SURBL (2761 hits): 20 (2%)
 Shared with URLDBL (152 hits): 57 (6%)
 Shared with WEIGHT15-19 (553 hits): 17 (1%)
 Shared with WEIGHT20 (18482 hits): 860 (98%)
 
 ---
 [This E-mail was scanned for viruses by Declude Virus
 (http://www.declude.com)]
 
 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.
 


[AUTOMATED NOTE: Your mail server