[Declude.JunkMail] Country-Chain filtering
X-Spam-Tests-Failed: DSBL, NJABLPROXIES, FIVETEN-SRC, COMBO-COUNTRY-US X-Country-Chain: ITALY-UNITED STATES-destination The testfile for COMBO-COUNTRY-US contains only one single line: COUNTRIES 0 STARTSWITH us Now the question is, how can this Country-Chain fail this test? We've in use v1.82 with Imail. Would it be possible to bether explain the country chain as it's not easy to send arround different test messages having all possible combinations of country chains? Whats the first entry, whats the last? What's the internal country chain, as we have to filter for us, it, fr, ... And not UNITED STATES, ITALY or FRANCE. Has this internal a different order (inversed)? Markus --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Country-Chain filtering
Markus, my foggy memory tells me that Country-Chain was designed to be US-centric, and is designed to trigger on suspicious routing for, say, US - Brazil - US. It wasn't designed to figure out the destination country and work backwards, nor was it designed to merely count the number of countries in the chain. If you get a better answer directly from Declude Support, please give us some feedback here. Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Markus Gufler Sent: Tuesday, March 29, 2005 3:20 AM To: Declude.JunkMail@declude.com Subject: [Declude.JunkMail] Country-Chain filtering X-Spam-Tests-Failed: DSBL, NJABLPROXIES, FIVETEN-SRC, COMBO-COUNTRY-US X-Country-Chain: ITALY-UNITED STATES-destination The testfile for COMBO-COUNTRY-US contains only one single line: COUNTRIES 0 STARTSWITH us Now the question is, how can this Country-Chain fail this test? We've in use v1.82 with Imail. Would it be possible to bether explain the country chain as it's not easy to send arround different test messages having all possible combinations of country chains? Whats the first entry, whats the last? What's the internal country chain, as we have to filter for us, it, fr, ... And not UNITED STATES, ITALY or FRANCE. Has this internal a different order (inversed)? Markus --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Country-Chain filtering
Andrew, You are thinking of the ROUTING test. That test shouldn't be used at all with servers located outside of the US. Let me try to explain the issue that Markus is seeing here. The COUNTRIES variable is something that is generated from the use of the all_list.dat file. This generates the variable COUNTRY and COUNTRIES. These variables contain lists of countries by their two letter code, with COUNTRY being the connecting server and COUNTRIES being the full list separated by spaces I believe. The %COUNTRYCHAIN% variable that is being used in the headers however is not exactly the same, and I have a feeling that it might be generated by data that is different from that contained in the all_list.dat, and if so, that would explain the issue. One could easily verify this by removing the all_list.dat and checking to see if the %COUNTRYCHAIN% variable was still populated in the headers. There is also the possibility that the header parsing is different for COUNTRIES/COUNTRY and %COUNTRYCHAIN%, so in this case COUNTRIES might be picking up a hop that %COUNTRYCHAIN% isn't. There is also of course a possibility of a bug or maybe an ordering issue. The STARTSWITH filter came along way after COUNTRIES was introduced, and Scott might not have bothered to order them properly since they couldn't be filtered with anything but CONTAINS at that time. It would be nice for someone from Declude to confirm the order and format of the COUNTRIES variable. ENDSWITH might well be the proper way to filter this for the first country in the chain. Unfortunately the only place that any of these things is documented is in the release notes. None of it appears in the manual, so I'm not even sure if %COUNTRYCHAIN% uses all_list.dat or not. Markus, if you were to share the full headers of this message, that would also help determine the source of the issue. Another note...since many zombie spammers forge headers prior to the connecting received header, it isn't always useful to know which country was first, but I don't assume to know exactly what you are doing with your filter so it may in fact be useful. The data also isn't always complete or accurate, and due to the way that IP space is used, it could never be perfectly accurate. Matt Colbeck, Andrew wrote: Markus, my foggy memory tells me that Country-Chain was designed to be US-centric, and is designed to trigger on suspicious routing for, say, US - Brazil - US. It wasn't designed to figure out the destination country and work backwards, nor was it designed to merely count the number of countries in the chain. If you get a better answer directly from Declude Support, please give us some feedback here. Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Markus Gufler Sent: Tuesday, March 29, 2005 3:20 AM To: Declude.JunkMail@declude.com Subject: [Declude.JunkMail] Country-Chain filtering X-Spam-Tests-Failed: DSBL, NJABLPROXIES, FIVETEN-SRC, COMBO-COUNTRY-US X-Country-Chain: ITALY-UNITED STATES-destination The testfile for COMBO-COUNTRY-US contains only one single line: COUNTRIES 0 STARTSWITH us Now the question is, how can this Country-Chain fail this test? We've in use v1.82 with Imail. Would it be possible to bether explain the country chain as it's not easy to send arround different test messages having all possible combinations of country chains? Whats the first entry, whats the last? What's the internal country chain, as we have to filter for us, it, fr, ... And not UNITED STATES, ITALY or FRANCE. Has this internal a different order (inversed)? Markus --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Country-Chain filtering
Ding! ROUTING was exactly what I was thinking of. lashback to the server roomlash Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Tuesday, March 29, 2005 1:04 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Country-Chain filtering Andrew, You are thinking of the ROUTING test. That test shouldn't be used at all with servers located outside of the US. Let me try to explain the issue that Markus is seeing here. The COUNTRIES variable is something that is generated from the use of the all_list.dat file. This generates the variable COUNTRY and COUNTRIES. These variables contain lists of countries by their two letter code, with COUNTRY being the connecting server and COUNTRIES being the full list separated by spaces I believe. The %COUNTRYCHAIN% variable that is being used in the headers however is not exactly the same, and I have a feeling that it might be generated by data that is different from that contained in the all_list.dat, and if so, that would explain the issue. One could easily verify this by removing the all_list.dat and checking to see if the %COUNTRYCHAIN% variable was still populated in the headers. There is also the possibility that the header parsing is different for COUNTRIES/COUNTRY and %COUNTRYCHAIN%, so in this case COUNTRIES might be picking up a hop that %COUNTRYCHAIN% isn't. There is also of course a possibility of a bug or maybe an ordering issue. The STARTSWITH filter came along way after COUNTRIES was introduced, and Scott might not have bothered to order them properly since they couldn't be filtered with anything but CONTAINS at that time. It would be nice for someone from Declude to confirm the order and format of the COUNTRIES variable. ENDSWITH might well be the proper way to filter this for the first country in the chain. Unfortunately the only place that any of these things is documented is in the release notes. None of it appears in the manual, so I'm not even sure if %COUNTRYCHAIN% uses all_list.dat or not. Markus, if you were to share the full headers of this message, that would also help determine the source of the issue. Another note...since many zombie spammers forge headers prior to the connecting received header, it isn't always useful to know which country was first, but I don't assume to know exactly what you are doing with your filter so it may in fact be useful. The data also isn't always complete or accurate, and due to the way that IP space is used, it could never be perfectly accurate. Matt Colbeck, Andrew wrote: Markus, my foggy memory tells me that Country-Chain was designed to be US-centric, and is designed to trigger on suspicious routing for, say, US - Brazil - US. It wasn't designed to figure out the destination country and work backwards, nor was it designed to merely count the number of countries in the chain. If you get a better answer directly from Declude Support, please give us some feedback here. Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Markus Gufler Sent: Tuesday, March 29, 2005 3:20 AM To: Declude.JunkMail@declude.com Subject: [Declude.JunkMail] Country-Chain filtering X-Spam-Tests-Failed: DSBL, NJABLPROXIES, FIVETEN-SRC, COMBO-COUNTRY-US X-Country-Chain: ITALY-UNITED STATES-destination The testfile for COMBO-COUNTRY-US contains only one single line: COUNTRIES 0 STARTSWITH us Now the question is, how can this Country-Chain fail this test? We've in use v1.82 with Imail. Would it be possible to bether explain the country chain as it's not easy to send arround different test messages having all possible combinations of country chains? Whats the first entry, whats the last? What's the internal country chain, as we have to filter for us, it, fr, ... And not UNITED STATES, ITALY or FRANCE. Has this internal a different order (inversed)? Markus --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type
Re: [Declude.JunkMail] Recip variables
John, I would think that with 2.0.5 and some other recent versions changing the variables according to the ROUTETO action, that one would expect to see some inconsistencies with those things. The behavior in 2.0.6 is supposed to change that back to the old way, or something similar to the old way, but they might not have thought about applying that to the variables that you listed. Needless to say, those variables are highly desirable to reflect the values prior to the actions being applied and not after modification by things like ROUTETO. Matt John Tolmachoff (Lists) wrote: If any one is seeing/having problems with the 3 recip variables please let Declude support know ASAP so that it can be properly fixed before the next version is released. %ALLRECIPS% %LOCALRECIPS% %REALRECIPS% John T eServices For You --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Country-Chain filtering
Here are the full headers Received: from hotmail.com [65.54.185.14] by mail.zcom.it with ESMTP (SMTPD32-8.15) id A6892E300A8; Mon, 28 Mar 2005 21:10:01 +0200 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Mon, 28 Mar 2005 11:09:16 -0800 Message-ID: [EMAIL PROTECTED] Received: from 82.48.60.139 by by15fd.bay15.hotmail.msn.com with HTTP; Mon, 28 Mar 2005 19:09:16 GMT From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] So the delivery-chain was Italian DSL = US Hotmail Server = MS demonstration that their SMTPSVC is working well ;-) = my own SMTP Server in Italy I've already had such problems with the country/ies test and asked Scott for an explanation. I believe it was in a period while Scott has had way too much work. What I want to do: I consider it not a good idea to assign weights to every mail comming from a certain country. But I discovered that it's a very good thing to combine the origin (or pass trough) of certain messages with different other content-, rule- or BL-based tests and so for example add extra points if the message is comming from country X and is also failing other tests. This way I can assign bigger weights to blacklisted DUL-IP's all over the world while bypassing this for messages comming from Italy, Austria, Germany and Switzerland as it would generate many FP's for legit messages. (This could be used also the other way with COUNTRIES END CONTAINS xx to list only trusted instead of hundreds of untrusted countries) All I have to know is exactly how this part of declude JM is working. As already said it's not so easy as other tests to simply try it out. Unfortunately I haven't MTA's placed all over the world. So please can someone who is working on this code try to explain what and how it's doing exactly. Thanks in advance Markus -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Tuesday, March 29, 2005 11:04 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Country-Chain filtering Andrew, You are thinking of the ROUTING test. That test shouldn't be used at all with servers located outside of the US. Let me try to explain the issue that Markus is seeing here. The COUNTRIES variable is something that is generated from the use of the all_list.dat file. This generates the variable COUNTRY and COUNTRIES. These variables contain lists of countries by their two letter code, with COUNTRY being the connecting server and COUNTRIES being the full list separated by spaces I believe. The %COUNTRYCHAIN% variable that is being used in the headers however is not exactly the same, and I have a feeling that it might be generated by data that is different from that contained in the all_list.dat, and if so, that would explain the issue. One could easily verify this by removing the all_list.dat and checking to see if the %COUNTRYCHAIN% variable was still populated in the headers. There is also the possibility that the header parsing is different for COUNTRIES/COUNTRY and %COUNTRYCHAIN%, so in this case COUNTRIES might be picking up a hop that %COUNTRYCHAIN% isn't. There is also of course a possibility of a bug or maybe an ordering issue. The STARTSWITH filter came along way after COUNTRIES was introduced, and Scott might not have bothered to order them properly since they couldn't be filtered with anything but CONTAINS at that time. It would be nice for someone from Declude to confirm the order and format of the COUNTRIES variable. ENDSWITH might well be the proper way to filter this for the first country in the chain. Unfortunately the only place that any of these things is documented is in the release notes. None of it appears in the manual, so I'm not even sure if %COUNTRYCHAIN% uses all_list.dat or not. Markus, if you were to share the full headers of this message, that would also help determine the source of the issue. Another note...since many zombie spammers forge headers prior to the connecting received header, it isn't always useful to know which country was first, but I don't assume to know exactly what you are doing with your filter so it may in fact be useful. The data also isn't always complete or accurate, and due to the way that IP space is used, it could never be perfectly accurate. Matt Colbeck, Andrew wrote: Markus, my foggy memory tells me that Country-Chain was designed to be US-centric, and is designed to trigger on suspicious routing for, say, US - Brazil - US. It wasn't designed to figure out the destination country and work backwards, nor was it designed to merely count the number of countries in the chain. If you get a better answer directly from Declude Support, please give us some feedback here. Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Markus
[Declude.JunkMail] MTLDB Hotmail
Hi; Am I imagining things or Declude.mtldb has listed a Hotmail ip address in the blacklist? X-RBL-Warning: [DECLUDE.ip4r.MTLDB]: "IP is listed in MTLDB" Spamcop has also that IP listed.. X-RBL-Warning: [SPAMCOP.ip4r]: "Blocked - see http://www.spamcop.net/bl.shtml?64.4.61.200" The IP: X-Note: Reverse DNS IP: bay102-dav3.bay102.hotmail.com [64.4.61.75] I am seeing a lot of emails with Hotmail being tagged.. Regards, Kami
Re: [Declude.JunkMail] Country-Chain filtering
Markus, One could set up a test by manually constructing several messages and switching around the IP's, but personally I couldn't justify doing that, so Declude's assistance with this would seem to be the best use of time. Based on the headers, I would suspect one of two things. First, the received line below the Message-ID is non-standard. It doesn't contain a bracketed IP address. It could be that the parsing of IP's for checking against all_list.dat is not catching this hop based on it either lying below the Message-ID, or due to the format of the IP in the received line. The other possibility would seem to be that the order in the COUNTRIES variable is reversed. The 82.48.60.139 is serviced by RIPE and not ARIN, and I would seem unlikely that the data in all_list.dat would indicate that this is from the US. Maybe you could requeue the E-mail (create a matching Q file and strip the Declude headers, placing the files in your spool and calling IMail1.exe with the Q file location as an argument) and change your filter to "IS" from "STARTSWITH" in your COMBO-COUNTRY-US. That would verify if there was only one value in the COUNTRIES variable. If the IS filter hits, that would suggest the first of the possibilities as being the issue. If it doesn't hit, that would suggest that the second possibility was more likely. Regarding what you wish to do, I think it might actually be better to use COUNTRY instead of COUNTRIES in this case. COUNTRY is only the connecting hop and it is never forged, so you would be testing the true source of the E-mail. It isn't nearly as useful to know the prior hops unless you want to detect anything along the way that came from a particular place by using a CONTAINS filter. Considering that many legitimate messages will have a private IP on the first hop, using a STARTSWITH filter won't do anything for you, and if it is zombie spam, it just simply can't be trusted because they are always single hop messages where anything more than that is forged. DUL tests should by default only be applied to the last hop, so using COUNTRY to indicate stuff from Italy, Germany, etc. would seem more appropriate. Then again, maybe I'm missing something, and if so, you don't need to explain yourself as I'm just trying to lend a little predictive advice. BTW, no doubt the DUL lists are highly inaccurate in non-North American countries. Most of my DUL false positives come from Asia where it seems they list first and exclude later, and I have seen them exclude only one IP when reported instead of the whole /24 or larger. It's a problem where admins in those countries have no clue that they might be listed, and the convoluted process of reporting such things to a blacklist probably makes the language barrier much larger (try reporting an FP to SORBS under their new system). Matt Markus Gufler wrote: Here are the full headers Received: from hotmail.com [65.54.185.14] by mail.zcom.it with ESMTP (SMTPD32-8.15) id A6892E300A8; Mon, 28 Mar 2005 21:10:01 +0200 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Mon, 28 Mar 2005 11:09:16 -0800 Message-ID: [EMAIL PROTECTED] Received: from 82.48.60.139 by by15fd.bay15.hotmail.msn.com with HTTP; Mon, 28 Mar 2005 19:09:16 GMT From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] So the delivery-chain was Italian DSL = US Hotmail Server = MS demonstration that their SMTPSVC is working well ;-) = my own SMTP Server in Italy I've already had such problems with the country/ies test and asked Scott for an explanation. I believe it was in a period while Scott has had way too much work. What I want to do: I consider it not a good idea to assign weights to every mail comming from a certain country. But I discovered that it's a very good thing to combine the origin (or pass trough) of certain messages with different other content-, rule- or BL-based tests and so for example add extra points if the message is comming from country X and is also failing other tests. This way I can assign bigger weights to blacklisted DUL-IP's all over the world while bypassing this for messages comming from Italy, Austria, Germany and Switzerland as it would generate many FP's for legit messages. (This could be used also the other way with "COUNTRIES END CONTAINS xx" to list only "trusted" instead of hundreds of "untrusted" countries) All I have to know is exactly how this part of declude JM is working. As already said it's not so easy as other tests to simply try it out. Unfortunately I haven't MTA's placed all over the world. So please can someone who is working on this code try to explain what and how it's doing exactly. Thanks in advance Markus -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Matt Sent: Tuesday, March 29, 2005 11:04 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Country-Chain filtering Andrew, You are thinking of the ROUTING test. That test
Re: [Declude.JunkMail] MTLDB Hotmail
We've been having problems with MTLDB doing the same thing...started around 8:00A PST (California, US). -- Original Message -- From: Kami Razvan [EMAIL PROTECTED] Reply-To: Declude.JunkMail@declude.com Date: Tue, 29 Mar 2005 17:06:14 -0500 Hi; Am I imagining things or Declude.mtldb has listed a Hotmail ip address in the blacklist? X-RBL-Warning: [DECLUDE.ip4r.MTLDB]: IP is listed in MTLDB Spamcop has also that IP listed.. X-RBL-Warning: [SPAMCOP.ip4r]: Blocked - see http://www.spamcop.net/bl.shtml?64.4.61.200; The IP: X-Note: Reverse DNS IP: bay102-dav3.bay102.hotmail.com [64.4.61.75] I am seeing a lot of emails with Hotmail being tagged.. Regards, Kami -- Kim W. Premuda FastWave Internet Services San Diego, CA -- --- [This E-mail scanned for viruses by Declude Virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] MTLDB Hotmail
Has the effectiveness of the MTLDB improved at all? I stopped using it months ago because of rampant false positives. - Original Message - From: Kami Razvan To: Declude.JunkMail@declude.com Sent: Tuesday, March 29, 2005 4:06 PM Subject: [Declude.JunkMail] MTLDB Hotmail Hi; Am I imagining things or Declude.mtldb has listed a Hotmail ip address in the blacklist? X-RBL-Warning: [DECLUDE.ip4r.MTLDB]: "IP is listed in MTLDB" Spamcop has also that IP listed.. X-RBL-Warning: [SPAMCOP.ip4r]: "Blocked - see http://www.spamcop.net/bl.shtml?64.4.61.200" The IP: X-Note: Reverse DNS IP: bay102-dav3.bay102.hotmail.com [64.4.61.75] I am seeing a lot of emails with Hotmail being tagged.. Regards, Kami
