Re: [Declude.JunkMail] Passing weight to Externalplus test
Scott,
FYI, in testing I found that the %WEIGHT% is being passed in, however
it seems to be 500 points higher than in reality, with all the weights
showing up as being between 500 and 600 over the space of my test.
Thanks,
Matt
Matt wrote:
Scott,
I've been playing with this for a bit now and it seems that the weight
isn't being passed as %WEIGHT%, or maybe it is strangely formatted.
My script now uses two values, the first being the current weight in
Declude, and the second being the SKIPIFWEIGHT equivalent. The
following line doesn't work (test never returns a result):
SIZE-S external 13 "cscript
C:\IMail\Declude\Size.vbs //NoLogo //T:2 %WEIGHT% 28" 0 0
However the following line does work (script always returns a result):
SIZE-S external 13 "cscript
C:\IMail\Declude\Size.vbs //NoLogo //T:2 10 28" 0 0
Here's the source of the Size.vbs file for reference:
If WScript.arguments(0) = WScript.arguments(1)
Then
WScript.Quit(0)
Else
Dim objFSO, objFile
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.GetFile(WScript.arguments(2))
If objFile.size 512 Then
WScript.Quit(11) 'SIZE-XXS [0 KB - 0.5 KB]
ElseIF objFile.size 1024 Then
WScript.Quit(12) 'SIZE-XS [0.5 KB - 1 KB]
ElseIF objFile.size 30720 Then
WScript.Quit(13) 'SIZE-S [1 KB - 30 KB]
ElseIF objFile.size 102400 Then
WScript.Quit(14) 'SIZE-M [30 KB - 100 KB]
ElseIF objFile.size 307200 Then
WScript.Quit(15) 'SIZE-L [100 KB - 300 KB]
ElseIF objFile.size 1024000 Then
WScript.Quit(16) 'SIZE-XL [300 KB - 1,000 KB]
ElseIf objFile.size = 1024000 Then
WScript.Quit(17) 'SIZE-XXL [1,000+ KB]
Else
WScript.Quit(0)
End If
Set objFile = nothing
Set objFSO = nothing
End If
Could you take a look at this when you get a chance.
Thanks,
Matt
R. Scott Perry wrote:
Is there another variable available like
%CURRENTWEIGHT% that could be used for this purpose (whatever
SKIPIFWEIGHT uses)?
There is now an interim 1.79i3 at http://www.declude.com/interim
that
changes the %WEIGHT% variable so that it will include the current
weight if it is used before the total weight is calculated.
-Scott
---
Declude JunkMail: The advanced anti-spam solution for IMail mailservers
since 2000.
Declude Virus: Ultra reliable virus detection and the leader in
mailserver vulnerability detection.
Find out what you've been missing: Ask for a free 30-day evaluation.
---
[This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
--
=
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=
--
=
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=
Re: [Declude.JunkMail] Passing weight to Externalplus test
Thanks :) R. Scott Perry wrote: FYI, in testing I found that the %WEIGHT% is being passed in, however it seems to be 500 points higher than in reality, with all the weights showing up as being between 500 and 600 over the space of my test. There is a new interim 1.79i4 that fixes this. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers since 2000. Declude Virus: Ultra reliable virus detection and the leader in mailserver vulnerability detection. Find out what you've been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Passing weight to Externalplus test
Cheer up :) No problem. Just wondered about the 8 minutes. :-) I know that in Declude we have a great tool and I can't have it 100% as I want. Hope your external test will work fine and you can add additional tests. As we check for message sizes in SpamChk for over a year now maybe I can give you some input about my observations. What about the idea to use this script as an external weight test and let return the script the result as weight? So you have one single test in the declude.cfg file and you can return whatever weight you want directly to the delcude weighting system. For example I've seen that around 50% of al incomming spam is under 5 kBytes. However there are spam messages up to 100 kBytes. (see attached diagram based on around 2 hold spam messages on our server in the last 4 days) Based on this values we've decided to give a very small negative weight to messages having less then 32 kByte. More negative points for messages having at least 48 kBytes and another more neg. points for messages having more then 64 kByte Theoreticaly it should be a good idea to return the result directly dependent on the file size. So for example the minimum file size for a negative weight should be 30 kByte. This should return e negative weight of 5% of the hold value. (-1 point for hold-on-20) The returned negative weight should be increased for every additional 10 kBytes by 5% of the hold weight. Size Weight 10 0 20 0 30 -1 40 -2 50 -3 60 -4 ... 100 -8 ... 220 -20 On my server I can see the following variation of message file sizes: 12% 64 kByte 2% 48 to 64 kByte 6%32 to 47 kByte 80% 32 kByte I consider negative points for large messages as relative secure because spammers - even if using an army of zombies - can't easily send out a large quantity of spam of this size. Markus spam_filesizes.pdf Description: Adobe PDF document
Re: [Declude.JunkMail] Passing weight to Externalplus test
On 7 Apr 2004 at 17:20, R. Scott Perry wrote: There is now an interim 1.79i3 at http://www.declude.com/interim that changes the %WEIGHT% variable so that it will include the current weight if it is used before the total weight is calculated. Scott, For me this is what makes me so loyal to your products. You listen to your customers.. -Nick Hayer -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers since 2000. Declude Virus: Ultra reliable virus detection and the leader in mailserver vulnerability detection. Find out what you've been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. [AUTOMATED NOTE: Your mail server [170.222.200.91] is missing a reverse DNS entry. All Internet hosts are required to have a reverse DNS entry. The missing reverse DNS entry will cause your mail to be treated as spam on some servers, such as AOL.] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Passing weight to Externalplus test
Markus, Thanks for the stats. I've actually been keeping copies of all of the false positives that we are reprocessing since Monday. Here's a break down by the sender (considering that some newsletters and ads are sent to multiple recipients and that throws off the numbers): 1 - 0.5 KB 1 - 0.5 KB to 1 KB 5 - 1 KB to 5 KB 2 - 5 KB to 10 KB 2 - 10 KB to 15 KB 6 - 15 KB to 20 KB 0 - 20 KB to 30 KB 1 - 30 KB to 40 KB 2 - 40 KB to 50 KB 0 - 50 KB to 75 KB 2 - 75 KB to 100 KB 1 - 100 KB to 200 KB 1 - 200 KB to 300 KB 1 - 300 KB I'm mostly concerned about false positives and performance currently, and while our FP rate is regularly below 0.02% now, this still takes almost as much time to find problems and fix them as it did when our rate was many times more that. I need to therefore balance the potential of causing FP's with adding points for weight with the incremental benefit of being able to block a small extra percentage of spam, and err heavily to the side of protecting from FP's. Also note that I am very liberal in classifying good E-mail, allowing through anything where the recipient has a first-party relationship with the sender. FootLocker.com for instance sent me two ads in a week for the first time since I bought something from them 20 months ago. I figure that as long as they honor my opt-out (despite not every opting-in to their ads), this protects those that want the content from having it blocked. Unfortunately many administrators consider this stuff to be spam, and it makes my job more difficult because of reports to SpamCop, Sniffer, and other places that nominate such things. While this stuff may be spam, people should also take note of the limitations of the blocking mechanism to differentiate between spam from a particular source, and a legitimate E-mail from that source or containing similar links. If you can't differentiate, administrators should seek out a better method IMO. Anyway... I've done some review of our held spam that scores between 10 and 24 points on our system (a 150% boundary) and for instance so far in the past 4 days every message held over 100 KB was a FP from an individual (the worst kind). There's definitely spam between 30 KB and 100 KB, but as a percentage, this also represents an area where messages falling in that range are far more likely to be a false positive because newsletters from dirty sources often enough come in over 30 KB, while opt-in spammers don't generally bother with that much content and zombie spammers certainly don't (for now at least). My thoughts about the weight test are two fold. For one, I'm really only interested in adding points to zombie spam since static spammers can be caught once and then their whole IP space can be blacklisted. Static spammers aren't very dynamic outside of their owned blocks, and I'm not very concerned about proactive protections using a message size filter. Zombie spam though is almost always below 5 KB, and sometimes below 0.5 KB. If I can narrow this down to 99.9% of it falling below a certain size, I can use the size test to defeat my processor intensive filters like GIBBERISH, IPLINKED and @LINKED among others. Yesterday I managed to skip processing these filters on 5% of my mail volume when set to only run below 30 KB in size. If that magic number is more like 5 KB, I can save much more in terms of processing power. Another added benefit is that when you don't run a filter on messages above a certain size, you limit the potential of a false positive with that filter. For instance, I see plenty of FP's on IPLINKED in newsletters, but this filter is built to target zombie spam, not spam from static sources which are easily tagged. So in effect, even without subtracting points, and just using larger sizes to defeat certain tests, this protects from FP's and saves processing power. So far I'm differentiating between filters built for static sources or a mix, and filters built specifically for zombie spam, and not processing those types according to different message sizes. I'm probably only going to add points to things below 0.5 KB, and this will only be 10% to 20% of my hold weight. I did see some FP's from 0.5 KB to 1 KB, mostly very brief messages that just scraped under the limit. I'm going to try looking for the minimum size of a message sent from a legit mail client and only add points below that point. The sweet spot for zombie spam certainly appears to be below 5 K, but I have to do some more research on that. Unfortunately I can't parse the COPYFILE message bodies for headers so that I could more effectively identify the zombie stuff. For those that have asked or are interested in the weight filter, what I'm going to do is set it up with the ability to set 7 different ranges by way of the arguments in a comma delimited string. This way everyone can tune it to their own needs. The skipping of the filter will also be configurable with arguments as long as you are using 1.79i4+. Matt Markus
Re: [Declude.JunkMail] Passing weight to Externalplus test
The %WEIGHT% is supposed to be passed into the script so that it can decide whether or not to fully run or immediately quit, but I can't get it to quit. Although this isn't critical for this one script, it is definitely the main component of the Sniffer bypassed that I would like to also put together. The problem here is that the %WEIGHT% variable isn't calculated until after all the tests are run. I don't believe there is a way to pass an external program the current weight of the E-mail at the time the test is run. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers since 2000. Declude Virus: Ultra reliable virus detection and the leader in mailserver vulnerability detection. Find out what you've been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Passing weight to Externalplus test
Scott, ...and all this time I was banking on this being possible. Is there another variable available like %CURRENTWEIGHT% that could be used for this purpose (whatever SKIPIFWEIGHT uses)? I recall Sandy releasing a SpamD port back in January that included at least the hooks for this, but I was under the impression that Declude supported it (my fault for assuming I guess). This isn't at all important for the Size test, but it would be impossible to create a bypass function for Sniffer based on weight if this wasn't available. Some 60%-70% of the messages hitting Sniffer are already well above my Drop weight, and with some work on a trusted local whitelist (in DNS), I could also skip this test (and others) if under a certain weight. I've been trying hard to solve my own issues where possible without asking for new functionality to Declude, but I'm afraid that I might have to again ask :) Thanks, Matt R. Scott Perry wrote: The %WEIGHT% is supposed to be passed into the script so that it can decide whether or not to fully run or immediately quit, but I can't get it to quit. Although this isn't critical for this one script, it is definitely the main component of the Sniffer bypassed that I would like to also put together. The problem here is that the %WEIGHT% variable isn't calculated until after all the tests are run. I don't believe there is a way to pass an external program the current weight of the E-mail at the time the test is run. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers since 2000. Declude Virus: Ultra reliable virus detection and the leader in mailserver vulnerability detection. Find out what you've been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Passing weight to Externalplus test
Is there another variable available like %CURRENTWEIGHT% that could be used for this purpose (whatever SKIPIFWEIGHT uses)? There is now an interim 1.79i3 at http://www.declude.com/interim that changes the %WEIGHT% variable so that it will include the current weight if it is used before the total weight is calculated. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers since 2000. Declude Virus: Ultra reliable virus detection and the leader in mailserver vulnerability detection. Find out what you've been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Passing weight to Externalplus test
There is now an interim 1.79i3 at WOW! I have to analyze Matt's and Sanford's messages/spelling/psycology. How the hell it's possible to have such a fast reaction (8 minutes!!!) for such a request? No doubt, support issues are resolved very fast. Also realy important things like EZIP. This is important and good. But I'm asking for month's now for simple new features that in the meantime was repeated several times by other customers... still waiting Can't imagine what I'm doing wrong here. Markus :-( --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Passing weight to Externalplus test
Did you send Scott a Christmas card? :) Jason -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Markus Gufler Sent: Wednesday, April 07, 2004 4:38 PM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Passing weight to Externalplus test There is now an interim 1.79i3 at WOW! I have to analyze Matt's and Sanford's messages/spelling/psycology. How the hell it's possible to have such a fast reaction (8 minutes!!!) for such a request? No doubt, support issues are resolved very fast. Also realy important things like EZIP. This is important and good. But I'm asking for month's now for simple new features that in the meantime was repeated several times by other customers... still waiting Can't imagine what I'm doing wrong here. Markus :-( --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Passing weight to Externalplus test
Markus, Just to be fair, I have mentioned or asked for a lot of different things that have not been introduced into Declude. Clearly by the speed of this modification, it was a very minor change to the environment, essentially exposing data that wasn't previously exposed in this way, but existed in other forms. Changing the way that Declude Virus handles per-domain settings though probably represents a major re-write to the system, and although I definitely want to see this, I have no expectations of it happening at least until after the next full release. There are other items also that appear that they may be minor modifications that also haven't been changed, and I'm sure that there is a reason for these, and although my opinion or perception may differ, I accept that it's Scott's call. I'm absolutely certain though that Scott is not playing favorites here. I can tell you that it took me a month and multiple posts to figure out why I couldn't get a VBScript to return a result code to Declude, and the preface for that functionality required the presence of a current weight that didn't exist to be passed to the script. I've spent probably 20 trying to figure out something that was not possible until a moment ago, and that's a bit frustrating honestly, but I am of course relieved now. This is also not functionality built for just me, it's for everyone because after the scripts are finished, I'm going to share them with everyone, and the benefit can be seen by anyone using any type of external test, for instance SpamD and SpamChk (if you enable it). If you add that together with the ease of the change, it makes perfect sense that he would at least consider this strongly. The majority of things that I have asked for or indicated interest in though have not been provided, but I ask for or indicate interest in many of these things just to show that there is at least one or one more person interested in them. I'm not unhappy though with the response; I'm definitely getting my money's worth and I hope that in return for the consideration for my multiple requests, that I am also providing something of value in return as many around here have as well. What I am also trying to do here I expect will someday be built into Declude (skipping external tests by weight, and having a test for message size), and in reality that's what I would have preferred, but because I didn't expect for my requests to be honored, I sought to do what I could on my own. So in reality, I've asked for skipping external tests by weight and didn't get it, and then I asked for a weight variable so that I could do this myself and got it. That seems to be par for the course. Cheer up :) Matt Markus Gufler wrote: There is now an interim 1.79i3 at WOW! I have to analyze Matt's and Sanford's messages/spelling/psycology. How the hell it's possible to have such a fast reaction (8 minutes!!!) for such a request? No doubt, support issues are resolved very fast. Also realy important things like EZIP. This is important and good. But I'm asking for month's now for "simple" new features that in the meantime was repeated several times by other customers... still waiting Can't imagine what I'm doing wrong here. Markus :-( --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re: [Declude.JunkMail] Passing weight to Externalplus test
Scott,
I've been playing with this for a bit now and it seems that the weight
isn't being passed as %WEIGHT%, or maybe it is strangely formatted.
My script now uses two values, the first being the current weight in
Declude, and the second being the SKIPIFWEIGHT equivalent. The
following line doesn't work (test never returns a result):
SIZE-S external 13 "cscript
C:\IMail\Declude\Size.vbs //NoLogo //T:2 %WEIGHT% 28" 0 0
However the following line does work (script always returns a result):
SIZE-S external 13 "cscript
C:\IMail\Declude\Size.vbs //NoLogo //T:2 10 28" 0 0
Here's the source of the Size.vbs file for reference:
If WScript.arguments(0) = WScript.arguments(1) Then
WScript.Quit(0)
Else
Dim objFSO, objFile
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.GetFile(WScript.arguments(2))
If objFile.size 512 Then
WScript.Quit(11) 'SIZE-XXS [0 KB - 0.5 KB]
ElseIF objFile.size 1024 Then
WScript.Quit(12) 'SIZE-XS [0.5 KB - 1 KB]
ElseIF objFile.size 30720 Then
WScript.Quit(13) 'SIZE-S [1 KB - 30 KB]
ElseIF objFile.size 102400 Then
WScript.Quit(14) 'SIZE-M [30 KB - 100 KB]
ElseIF objFile.size 307200 Then
WScript.Quit(15) 'SIZE-L [100 KB - 300 KB]
ElseIF objFile.size 1024000 Then
WScript.Quit(16) 'SIZE-XL [300 KB - 1,000 KB]
ElseIf objFile.size = 1024000 Then
WScript.Quit(17) 'SIZE-XXL [1,000+ KB]
Else
WScript.Quit(0)
End If
Set objFile = nothing
Set objFSO = nothing
End If
Could you take a look at this when you get a chance.
Thanks,
Matt
R. Scott Perry wrote:
Is there another variable available like
%CURRENTWEIGHT% that could be used for this purpose (whatever
SKIPIFWEIGHT uses)?
There is now an interim 1.79i3 at http://www.declude.com/interim that
changes the %WEIGHT% variable so that it will include the current
weight if it is used before the total weight is calculated.
-Scott
---
Declude JunkMail: The advanced anti-spam solution for IMail mailservers
since 2000.
Declude Virus: Ultra reliable virus detection and the leader in
mailserver vulnerability detection.
Find out what you've been missing: Ask for a free 30-day evaluation.
---
[This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
--
=
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=
