> I've considered this a few times, every time I prepare to 
> suggest it I remember what happened with my idea to test for 
> long subjects, there just isn't enough uniformity.

Well. Maybe my idea is expressed from "the wrong side".
Watching the diagram I can also simply fathom that my current hold
weight is a little bit too low.
After adding some new SpamChk tests (we are currently testing) and some
new RBL-lists, the average value has increased a little bit. So the only
thing I have to do is to increase slightly the hold weight (or decrease
the points for every single test)

Remains the fact, that only 13% of our FP's whas recieved out of
business time. If there is some way to detect the senders local current
time or timezone this for sure will help again to reduce false positives
or false negatives using a "time-dependently hold weight"
 

> BTW, the graph is amazing, how is it made?

Hmmm, it's not an "out of the box" tool, but maybe someone can develop
it. I think it should be very easy but at the moment I'm not familiar
with any RAD tool...

So here the steps what I've done:

1.) grep all lines from the declude logfile containing "Total weight ="
Grep.exe is part of the unixtools what you can find on
http://unxutils.sourceforge.net/
Don't fear to "install" this tools. You can also simply extract the
zip-archive.

C:\imail\spool\grep -U "Total weight =" dec0624.log >
c:\imail\spool\tw0624.log

This will create a new file tw0624.log in the spool folder containing
only the lines with the total weight of any message processed by declude
junkmail.

Note: You need at least loglevel MID to see the "Total weight" lines in
the logfile.

2.) Now I've "elaborated" my tw-file 
In the following original line
06/21/2003 00:01:42 Q843b181400780c01 HELOBOGUS:19 .  Total weight = 19

a.) delete the date "06/21/2003 " 
00:01:42 Q843b181400780c01 HELOBOGUS:19 .  Total weight = 19

b.) replace the " Q" after the time with ";"
00:01:42;843b181400780c01 HELOBOGUS:19 .  Total weight = 19

c.) replace the "Total weight = " with ";"
00:01:42;843b181400780c01 HELOBOGUS:19 .  ;19

3.) Now you have a CSV file with the time in the first and the weight in
the third column.
You can import this for example into MS Excel

4.) To "decode" the HH:MM:SS time format in something usable for a
diagramm I've used the following formula:
C1 = (HOUR(A1)*3600)+(MINUTE(A1)*60)+SECONDS(A1)

This will give you in cell C1 the timecode in seconds

5.) Now you can play around with different diagrams, ...
For example you can also sort all rows by the weight to create a graph
like them attached to this message.
This will show you if you have done a good job configuring the tests so
that in the critical zone between 80 and 120% of your hold weight there
are minimal messages. (high slope)

I know looks like a lot of work, but it's done in few minutes and will
give you a great view what's going on on your junkmail filter.

All of this steps can be automizzed, if someone has time and knowledge
to create a small reporting tool...

Markus


<<attachment: returncodes_sorted.gif>>

Reply via email to