Scott,
This is _exactly_ what I have wanted Declude to do for over two years
now all the way down to the spec. This would add low and high skip
weight functionality to all filters, external apps and anything else
that might be skippable, and do so by using the config in such a way
that could easily be backwards compatible when not specified (you would
assume null null to be 0 0 and consider that disabled).
This would save tremendously in loading filter files and launching
external apps. I for instance have this functionality in one of my
external apps, but it still has to be run in order for the weight
skipping mechanism to operate. Other external apps have no weight
skipping built into them and this would add the much needed
functionality to save resources.
Matt
Scott Fisher wrote:
I've been mulling this one over as I watch my spam filtering CPU time
slowly taking over the email server. And I don't expect the number of
emails to go down.
For external programs and filters I think it would be a good idea to
add two optional fields to the global.cfg definition line: a minweight
and a maxweight. These would be the last two arguments and optional so
existing configs would not need to be changed.
For an external program:
INV-URIBL external 25 "D:\INVURIBL.exe %WEIGHT% %REMOTEIP%" 25 0
would become
INV-URIBL external 25 "D:\INVURIBL.exe %WEIGHT% %REMOTEIP%" 25 0
-50 300
in this case invuribl would only get run if the current weight was
between -50 and 300.
For a filter:
ATTACHMENT-GIF filter D:\ATTACHMENT-GIF.txt x 0 0
would become
ATTACHMENT-GIF filter D:\ATTACHMENT-GIF.txt x 0 0 -50 300
in this case the attachment-gif filter would only get processed if the
current weight was between -50 and 300
Here's why I think this is a good idea:
Declude could check the weights before launching the external program.
If it is over/under weight the external program would not be launched.
2 if statements to avoid launching a program. That seems like a CPU
time saver. Especially when multiplied by 10,000s of emails per day.
I use 6 external programs. I believe over half of the program launches
would be avoided because of stuff that has already been declared
obvious ham or obvious spam.
My final of the 6 programs, gets weight skipped over 90% of the time.
At 10,000 emails a day, avoiding 50% of the external programs would
save 30,000 program launches a day. I believe my 50% to be a
conservative number and I think that the percentage would average out
to be even higher.
Now I have about one hundred filters. The vast majority of them get
triggered with the skipweight since the email is already at a high
spam weight by the time it reaches the filters.
But still every one of these filter files needs to be opened, read and
closed for every email.
Again 2 IF statements per filter could avoid opening 100 files. That
seems to me to be a CPU time saver.
By the time, email reaches the filters, I think 75% of it is bypassing
filters by being over the skipweight. At 10,000 emails a day (small to
many of us). That would mean 750,000 filter files a day would not need
to be open, read and closed.
From the programming side, I don't believe the coding changes to be
too difficult. Weight verification/processing code already exists in
the Declude program. It would just need to be relocated.
I'm a pretty small user here, getting about 14,000 spams on a weekday.
Imagine the potential CPU savings for scaling this up to an ISP with
100,000 emails per day.
I don't know if this would have an impact on saving my CPU or not, but
it has to help even if it is a little.
Please consider this.
-----------------------------------------------------
Scott Fisher
Director of IT
Farm Progress Companies
191 S Gary Ave
Carol Stream, IL 60188
630-462-2323
This email message, including any attachments, is for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please contact
the sender by reply email and destroy all copies of the original
message. Although Farm Progress Companies has taken reasonable
precautions to ensure no viruses are present in this email, the
company cannot accept responsibility for any loss or damage arising
from the use of this email or attachments.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
