RE: [Declude.JunkMail] New test request
How about a test like this: NUMBERSINMAILFROM It would be similar to SUBJECTSPACES but would count the amount of numbers in the mail from address. You could then configure it for say if 10 or more, add 5 to the weight and so forth. John, We already look for sender-addresses containing more then 4 (SenderWithCodeMaybe) or more then 8 digits (SenderWithCode). So we count around 75% of spam-senders and 25% of FPs. As Scott sayd there are a lot of tipical Freemailer-Addresses like [EMAIL PROTECTED] creating FPs with such a test. But there are also auto-generated mailings having a sender address like [EMAIL PROTECTED] On a tipical day we can see around 10% of all incomming messages having between 4 and 7 digits. Other ~8% of incomming messages has more then 8 digits. It's not the best but a definitively usefull test in a weighting system. Markus --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] New test request
Hi; I have been following this discussion and it seems like for weight test it would be good. Some observations that could complement this: 1: Mailing list email addresses are long. I have not seen autogenerated addresses that are less than 10 or so characters. E.g. [EMAIL PROTECTED] [64.241.105.8] [EMAIL PROTECTED] But on the other hand spam like emails are typically about 10 or so characters. I think it is worth looking into John's suggestion with a consideration of the UserID length. E.g. from last night logs: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] I think we can use the length of the UserID to our advantage in implementing this test. 2: I wish we could run tests on UserID and domain separately. It seems like it would be much easier if the domain could be separated from the UserID since for example one could test for two dashes (--) in the domain. We are getting more more spam like hot--stuff.com Regards, Kami -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Markus Gufler Sent: Thursday, September 11, 2003 7:16 AM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] New test request How about a test like this: NUMBERSINMAILFROM It would be similar to SUBJECTSPACES but would count the amount of numbers in the mail from address. You could then configure it for say if 10 or more, add 5 to the weight and so forth. John, We already look for sender-addresses containing more then 4 (SenderWithCodeMaybe) or more then 8 digits (SenderWithCode). So we count around 75% of spam-senders and 25% of FPs. As Scott sayd there are a lot of tipical Freemailer-Addresses like [EMAIL PROTECTED] creating FPs with such a test. But there are also auto-generated mailings having a sender address like [EMAIL PROTECTED] On a tipical day we can see around 10% of all incomming messages having between 4 and 7 digits. Other ~8% of incomming messages has more then 8 digits. It's not the best but a definitively usefull test in a weighting system. Markus --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] New test request
Any thoughts, good or bad? John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of John Tolmachoff (Lists) Sent: Tuesday, September 09, 2003 10:32 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] New test request How about a test like this: NUMBERSINMAILFROM It would be similar to SUBJECTSPACES but would count the amount of numbers in the mail from address. You could then configure it for say if 10 or more, add 5 to the weight and so forth. John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] New test request
Any thoughts, good or bad? It's one that we do hope to add. It's not foolproof (such as [EMAIL PROTECTED]), but would be useful in helping catch spam. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you have been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] New test request
That would work great at detecting old Compuserve accounts :) I'm not convinced that this would be a very clear marker for spam though (depends on what the automated real stuff does), but you could probably set up a filter to test the theory First create a filter file test and score it as a negative 2: SENDERNUM filter C:\IMail\Declude\SenderNum.txt x -2 0 Then fill the file with an entry for numbers 10-99, scoring each one as a single point: MAILFROM 1 CONTAINS 10 MAILFROM 1 CONTAINS 11 MAILFROM 1 CONTAINS 12 ... This would score the number of digits in succession as follows, note that it will score higher if the address has numbers surrounded by letters, and lower if it is only numbers: 1 num = N/A 2 num = -1 3 num = 0 4 num = 1 5 num = 2 6 num = 3 7 num = 4 8 num = 5 9 num = 6 10 num = 7 ... Obviously there are two primary problems with this approach. First, it can have up to 86 points if the string of numbers is long enough (too bad you can't cap the total score of the filter). Secondly, it benefits senders by one point with just 3 successive numbers in their address. I'm thinking that some autoreply/auto-ticket systems might trip this filter though if they use the address instead of something in the subject line to track a communication. This might be same type of reason that some spammers use this...they might be cleaning their list with the bounces that get through HELO??? Who knows, maybe it's worth a try if you are really that interested in exploring whether or not the real thing would work??? Real-people E-mail shouldn't be failing too many other tests, and the automated stuff suffers greatly. Maybe having 3 numbers only in an E-mail address is something that rarely happens with spam??? Matt John Tolmachoff (Lists) wrote: Any thoughts, good or bad? John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED]] On Behalf Of John Tolmachoff (Lists) Sent: Tuesday, September 09, 2003 10:32 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] New test request How about a test like this: NUMBERSINMAILFROM It would be similar to SUBJECTSPACES but would count the amount of numbers in the mail from address. You could then configure it for say if 10 or more, add 5 to the weight and so forth. John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com
RE: [Declude.JunkMail] New test request
Sorry, I've no great insight on the positive uses of this test, but I can point out another exception. E-mail enabled pagers and RIM Blackberries often have their phone number as the e-mail address @TheProviderDomain.com instead of or in addition to the subscriber's name. Andrew. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] New test request
Title: Message maybe a bad idea - We send out e-mail that has a Variable Return Address, so that we can handle bounces well. In our case, that address is a combo of letters and numbers (lots of numbers sometimes). And, we work hard to make sure our mail is all requested! Other legit mailers use something similar. It does suggest the mail comes from a mailing list, but doesn't help to separate legit from spam. Rob www.iGive.com not convinced that this would be a very clear marker for spam though (depends on what the automated real stuff does), but you could probably set up a filter to test the theoryFirst create a filter file test and score it as a negative 2:
RE: [Declude.JunkMail] New test request
OK, my suggested weights are too high. Remember, the point of this test is to be used in the weighting system only. Pagers have 10 numbers, so I would actually start at either 11 or 15. An old CompuServe address will most likely not be failing other tests to where this one would put it over. How many numbers do those addresses have in them? I am thinking say if 11 numbers, add weight of 5. If 20 numbers, then add 15 more. John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Colbeck, Andrew Sent: Wednesday, September 10, 2003 12:32 PM To: '[EMAIL PROTECTED]' Subject: RE: [Declude.JunkMail] New test request Sorry, I've no great insight on the positive uses of this test, but I can point out another exception. E-mail enabled pagers and RIM Blackberries often have their phone number as the e-mail address @TheProviderDomain.com instead of or in addition to the subscriber's name. Andrew. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] New test request
Here's some examples of mailing lists that have lots of numbers (and letters) in the MAILFROM. You may find that you'll have to put in a counterweight everytime a user reports that they're missing mail when they sign up for a newsletter. Andrew 8) p.s. I've deliberately munged the addresses a little to make sure that our actual recpients won't get their newsletter interfered with because it was posted to a public forum. [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: [Declude.JunkMail] New test request
Dan Patnode wrote: Good point, The goal then should be to differentiate numbers used as codes from numbers used to confuse. The former tend to be contiguous while the later (in my experience), tend to be mixed in with letters. Perhaps if the test counted numbers with letters on both sides? Dan If you are looking for gibberish, look to the subject line and not the sender. I actually have a decent test for this in the subject line (don't use it in the body). The only false positives would come from very strange acronyms and auto-generated code such as tracking/receipt numbers. This scores higher the more gibberish you catch. It's been safe so far for me. GIBBERISHSUBfilterC:\IMail\Declude\GibberishSub.txt x10 SUBJECT2CONTAINSqb SUBJECT2CONTAINSqc SUBJECT2CONTAINSqd SUBJECT2CONTAINSqe SUBJECT2CONTAINSqf SUBJECT2CONTAINSqg SUBJECT2CONTAINSqh SUBJECT2CONTAINSqi SUBJECT2CONTAINSqj SUBJECT2CONTAINSqk SUBJECT2CONTAINSqm SUBJECT2CONTAINSqn SUBJECT2CONTAINSqo SUBJECT2CONTAINSqp SUBJECT2CONTAINSqr SUBJECT2CONTAINSqs SUBJECT2CONTAINSqt SUBJECT2CONTAINSqv SUBJECT2CONTAINSqx SUBJECT2CONTAINSqy SUBJECT2CONTAINSqz SUBJECT2CONTAINSvq SUBJECT2CONTAINSwq SUBJECT2CONTAINStq SUBJECT2CONTAINSjq SUBJECT2CONTAINSxd SUBJECT2CONTAINSxj SUBJECT2CONTAINSxk SUBJECT2CONTAINSxr SUBJECT2CONTAINSxz SUBJECT2CONTAINSzb SUBJECT2CONTAINSzc SUBJECT2CONTAINSzf SUBJECT2CONTAINSzj SUBJECT2CONTAINSzk SUBJECT2CONTAINSzl SUBJECT2CONTAINSzm SUBJECT2CONTAINSzx --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] New test request
JT Pagers have 10 numbers, so I would actually start at either 11 or 15. JT An old CompuServe address will most likely not be failing other tests to JT where this one would put it over. How many numbers do those addresses have JT in them? Nine digits, e.g [EMAIL PROTECTED] (that was mine for 5 years before they really had an Internet gateway...) Andrew 8) --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] New test request
I wouldn't consider that to be spam. Amazon? Travelocity? Yahoo Groups? Most of these are opt-in sources (by way of membership or purchase), and doing the bounce test that they are doing is in fact responsible use of commercial E-mail. If you are going to monitor for failed receivers, that means that your server isn't moving and you become a static target for the lists and heuristic filters. It's too bad that everyone doesn't do this. I'd much rather have a filter that detects no displayable text, or only searches decoded-non-HTML body text. Testing for that stuff would be a negative weight on my system...that's the F-P type of stuff that I'm trying to solve. Matt Colbeck, Andrew wrote: Here's some examples of mailing lists that have lots of numbers (and letters) in the MAILFROM. You may find that you'll have to put in a counterweight everytime a user reports that they're missing mail when they sign up for a newsletter. Andrew 8) p.s. I've deliberately munged the addresses a little to make sure that our actual recpients won't get their newsletter interfered with because it was posted to a public forum. [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: [Declude.JunkMail] New test request
MB GIBBERISHSUB filter C:\IMail\Declude\GibberishSub.txt x 1 0 MB SUBJECT2CONTAINSqb (snip) This looks good, Matthew. The weight is low enough to be cautious, and I suspect the only false positives you will get are on subject lines with that raw =?ISO-8859-1?B?UmU6U2lsZG stuff. (For those new to the party, Scott confirmed earlier that with declude.exe v1.75 (and a JunkMail Pro licence) these (8-bit encoded?) subject lines are not decoded to US-ASCII before applying a SUBJECT text match. Andrew 8) --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] New test request
In your examples, I only see 4 that would be FP under this, the ones from microsoft.com, unitiedmedia.com, yahoo groups, and Travelocity.com. newsletters.microsoft.com is already in a whitefilter. Yahoo groups are already in a whitefilter for known problems. Travelocity is a legit company, and therefore could go in a whitefilter. comicsmail.unitedmedia.com is something that can go into a whitefilter. The point is, someone can always come up with examples of how it can be used and how it would cause problems. Maybe it means at 15 add 5 and at 25 add another 10. John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Colbeck, Andrew Sent: Wednesday, September 10, 2003 1:35 PM To: '[EMAIL PROTECTED]' Subject: RE: [Declude.JunkMail] New test request Here's some examples of mailing lists that have lots of numbers (and letters) in the MAILFROM. You may find that you'll have to put in a counterweight everytime a user reports that they're missing mail when they sign up for a newsletter. Andrew 8) p.s. I've deliberately munged the addresses a little to make sure that our actual recpients won't get their newsletter interfered with because it was posted to a public forum. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] New test request
Thanks Andrew...I like my apples :) Some stuff could be put back in that I took out while testing the filter for the body before I found out that it caught attachments. I was careful to take out things like ql because of MSSQL, and I searched a dictionary file for matches on the other strings and deleted as was necessary, but other deletions were for more obscure reasons. My only concern was tagging an auto-generated serial/tracking number from an online receipt, but those should be generally numbers from looking over what I have saved from my purchases. I've gone kind of filter crazy in the last week. Anytime I see a message that should of been rejected, I look it over for patterns to match :) It's really too bad that this same filter doesn't work on the body text exclusively...that would tag a lot of the stuff that gets through. Matt Colbeck, Andrew wrote: MB GIBBERISHSUB filter C:\IMail\Declude\GibberishSub.txt x 1 0 MB SUBJECT2CONTAINSqb (snip) This looks good, Matthew. The weight is low enough to be cautious, and I suspect the only false positives you will get are on subject lines with that raw =?ISO-8859-1?B?UmU6U2lsZG stuff. (For those new to the party, Scott confirmed earlier that with declude.exe v1.75 (and a JunkMail Pro licence) these (8-bit encoded?) subject lines are not decoded to US-ASCII before applying a SUBJECT text match. Andrew 8) --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] New test request
Wow, what a sweet idea Matthew! Applying rules of English (like Q is always followed by U) to look for gibberish. :) Yea, so long as BODY searches attachments, any small code will sooner or later show up in an attachment. I've even had problems trying hard tests for complete words where an L was replaced with an I and it showed up in attachment PDF code. Dan On Wednesday, September 10, 2003 13:36, Matthew Bramble [EMAIL PROTECTED] wrote: Dan Patnode wrote: Good point, The goal then should be to differentiate numbers used as codes from numbers used to confuse. The former tend to be contiguous while the later (in my experience), tend to be mixed in with letters. Perhaps if the test counted numbers with letters on both sides? Dan If you are looking for gibberish, look to the subject line and not the sender. I actually have a decent test for this in the subject line (don't use it in the body). The only false positives would come from very strange acronyms and auto-generated code such as tracking/receipt numbers. This scores higher the more gibberish you catch. It's been safe so far for me. GIBBERISHSUBfilterC:\IMail\Declude\GibberishSub.txt x10 SUBJECT2CONTAINSqb SUBJECT2CONTAINSqc SUBJECT2CONTAINSqd SUBJECT2CONTAINSqe SUBJECT2CONTAINSqf SUBJECT2CONTAINSqg SUBJECT2CONTAINSqh SUBJECT2CONTAINSqi SUBJECT2CONTAINSqj SUBJECT2CONTAINSqk SUBJECT2CONTAINSqm SUBJECT2CONTAINSqn SUBJECT2CONTAINSqo SUBJECT2CONTAINSqp SUBJECT2CONTAINSqr SUBJECT2CONTAINSqs SUBJECT2CONTAINSqt SUBJECT2CONTAINSqv SUBJECT2CONTAINSqx SUBJECT2CONTAINSqy SUBJECT2CONTAINSqz SUBJECT2CONTAINSvq SUBJECT2CONTAINSwq SUBJECT2CONTAINStq SUBJECT2CONTAINSjq SUBJECT2CONTAINSxd SUBJECT2CONTAINSxj SUBJECT2CONTAINSxk SUBJECT2CONTAINSxr SUBJECT2CONTAINSxz SUBJECT2CONTAINSzb SUBJECT2CONTAINSzc SUBJECT2CONTAINSzf SUBJECT2CONTAINSzj SUBJECT2CONTAINSzk SUBJECT2CONTAINSzl SUBJECT2CONTAINSzm SUBJECT2CONTAINSzx --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
[Declude.JunkMail] New test request
How about a test like this: NUMBERSINMAILFROM It would be similar to SUBJECTSPACES but would count the amount of numbers in the mail from address. You could then configure it for say if 10 or more, add 5 to the weight and so forth. John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
