Understood on the DNS servers... I'll reconfigure that as soon as the blacklist functionality appears to be working again - we don't need any additional variables to troubleshoot.
We're still getting zero hits - here's an IP address which was listed on 12 blacklists, including Spamcop, SORBS, Barracuda and Spamhaus Zen: Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on bogons.cymru.com for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on bb.barracudacentral.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on smtp.dnsbl.sorbs.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on bl.spamcop.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on zen.spamhaus.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on psbl.surriel.com for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on db.wpbl.info for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on ix.dnsbl.manitu.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on l2.apews.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on combined.njabl.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on safe.dnsbl.sorbs.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on dsn.rfc-ignorant.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on block.stopspam.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on dnsbl.stopspam.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on hostkarma.junkemailfilter.com for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on dnsbl-1.uceprotect.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on dnsbl-2.uceprotect.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on dnsbl-3.uceprotect.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on dnsrbl.swinog.ch for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on cbl.abuseat.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on bl.spameatingmonkey.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.220.220[:53] on bl.mailspike.net for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Sending DNS-query to 208.67.222.222[:53] on bl.spamcannibal.org for RBL checks on 173.232.105.40; Jul-16-14 13:43:01 Commencing RBL checks on '173.232.105.40'; Jul-16-14 13:43:01 Got 0 answers, 0 replies and 0 hits after 0 seconds for RBL checks on '173.232.105.40'; Jul-16-14 13:43:01 Completed RBL checks on '173.232.105.40'; Spam is now beginning to trickle in as the cached entries expire. - Phil -----Original Message----- From: Phil Quesinberry Sent: Monday, July 14, 2014 12:55 PM To: 'For Users of ASSP' Subject: Re: [Assp-user] Inconsistent DNSBL scoring This one runs and appears to function properly except for the fact that we now get absolutely no blacklist hits unless they're cached. Putting DNSBL logging in verbose mode shows ASSP checking against the blacklists but there's never a hit. I went back and checked a suspicious IP which ASSP said had 0 hits and it was on SpamHaus Zen. Now it's always possible that it could have been added in the interim but I checked it within 5 minutes of getting the message. I'll continue to monitor to be sure and let you know if anything changes but there have been no real-time DNSBL or URIBL hits with the new version, only cached hits. Here's an excerpt from the log for the IP which appeared on SpamHaus Zen: Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.222.222[:53] on bogons.cymru.com for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.220.220[:53] on bogons.cymru.com for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.222.222[:53] on bb.barracudacentral.org for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.220.220[:53] on bb.barracudacentral.org for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.222.222[:53] on smtp.dnsbl.sorbs.net for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.220.220[:53] on smtp.dnsbl.sorbs.net for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.222.222[:53] on bl.spamcop.net for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.220.220[:53] on bl.spamcop.net for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.222.222[:53] on zen.spamhaus.org for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.220.220[:53] on zen.spamhaus.org for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.222.222[:53] on psbl.surriel.com for RBL checks on 23.236.229.249; Jul-14-14 12:17:09 Sending DNS(TXT)-query to 208.67.220.220[:53] on psbl.surriel.com for RBL checks on 23.236.229.249; ... Jul-14-14 12:17:09 Commencing RBL checks on '23.236.229.249'; Jul-14-14 12:17:09 Got 0 answers, 0 replies and 0 hits after 0 seconds for RBL checks on '23.236.229.249'; Jul-14-14 12:17:09 Completed RBL checks on '23.236.229.249'; - Phil ----- Re: [Assp-user] Inconsistent DNSBL scoring From: Thomas Eckardt - 2014-07-12 05:57:00 Sorry - there was another big BUG in V1. All defined configuration variables in the assp.cfg were imported - I used the one from V2 for compiling V1, so I don't saw these compiling errors. The corrected version 14189 is available at SF-CVS. Thomas -----Original Message----- From: Phil Quesinberry Sent: Friday, July 11, 2014 1:25 PM To: 'For Users of ASSP' Subject: Re: [Assp-user] Inconsistent DNSBL scoring Thanks Thomas. I downloaded the new code but when trying to run the new version I get the following errors: Global symbol "$rblTestMode" requires explicit package name at assp.pl line 16903. Global symbol "$switchSpamLoverToScoring" requires explicit package name at assp.pl line 16907. Global symbol "$switchTestToScoring" requires explicit package name at assp.pl line 16910. Global symbol "$RBLError" requires explicit package name at assp.pl line 17071. Global symbol "$rblTestMode" requires explicit package name at assp.pl line 17075. Global symbol "$rblTestMode" requires explicit package name at assp.pl line 17075. Global symbol "$RBLError" requires explicit package name at assp.pl line 17202. Global symbol "$ForceRBLCache" requires explicit package name at assp.pl line 17206. Global symbol "$rblTestMode" requires explicit package name at assp.pl line 17209. Global symbol "$rblTestMode" requires explicit package name at assp.pl line 17209. Global symbol "$switchSpamLoverToScoring" requires explicit package name at assp.pl line 17336. Global symbol "$switchTestToScoring" requires explicit package name at assp.pl line 17341. Global symbol "$uriblTestMode" requires explicit package name at assp.pl line 17342. Global symbol "$URIBLError" requires explicit package name at assp.pl line 17650. Global symbol "$uriblTestMode" requires explicit package name at assp.pl line 17652. Global symbol "$uriblTestMode" requires explicit package name at assp.pl line 17656. BEGIN not safe after errors--compilation aborted at assp.pl line 21442. I really don't know Perl but I'm guessing those variables aren't declared in V1? - Phil ----- Re: [Assp-user] Inconsistent DNSBL scoring From: Thomas Eckardt <Thomas.E<ckardt@th...> - 2014-07-11 06:36:45 I've implemented the complete V2 RBL and URIBL code in to 1.9.9 14188 - available at http://assp.cvs.sourceforge.net/viewvc/assp/asspV1/ Read the changed GUI for the ServiceProvider definitions for both checks !!!! Sorry for the very complex definitions - but this code was never written for V1 users and Fritz was adviced to not put this code in to V1. I'm unable to test this - because V1 does not meet my requirements - how ever, this code is running fine in V2. Thomas -----Original Message----- From: Phil Quesinberry Sent: Thursday, July 10, 2014 3:58 PM To: 'For Users of ASSP' Subject: Re: [Assp-user] Inconsistent DNSBL scoring Thomas, Crap... that explains a lot. Thanks for the heads up. If it's helpful at all, this stopped working properly when it started exhibiting the "new and improved!" behavior of the two-tiered Neutral/Fail approach where the scoring result from the first round of hits effectively serves as an index or lookup into the Neutral/Fail (30/50 etc.) scoring table. There appear to be a bunch of DNSBL-related changes between 1.9.9 (14059) and (14060) so that may be when this mess began. This problem appears to affect URIBL scoring as well. Cheers, - Phil -----Original Message----- From: Thomas Eckardt Sent: Thursday, July 10, 2014 1:26 PM To: ph...@qsystemsengineering.com; For Users of ASSP Subject: Re: [Assp-user] Inconsistent DNSBL scoring I had just a look in to the V1 code. The RBL code is totaly scrambled (mixed V2 and V1). hostkarma.junkemailfilter.com=>127.0.0.2=>45 This is a V2 syntax - it is accepted by V1 - but not used in the check. I don't know if I'll have the time to fix this wild code. How ever, in your example the return code was '127.0.1.1' , which has no weight assigned - so the default score is used. The RBL check will not work like expected ! Thomas Von: "Phil Quesinberry" <ph...@qsystemsengineering.com> An: <assp-user@lists.sourceforge.net> Datum: 10.07.2014 18:46 Betreff: Re: [Assp-user] Inconsistent DNSBL scoring After updating to 1.9.9 (14187), this appears to be working properly for cached entries but is still failing for non-cached entries. I don't think anything has changed, I'm pretty sure it worked properly for cached entries before. In the first example below, hostkarma.junkemailfilter.com hasn't even been assigned a weight for 127.0.1.1 results, the entries for that DNSBL are as follows: hostkarma.junkemailfilter.com=>127.0.0.2=>45 hostkarma.junkemailfilter.com=>127.0.1.2=>35 hostkarma.junkemailfilter.com=>127.0.0.4=>30 >From the log: Jul-10-14 03:09:49 m-40497-02320 [SSL-out] 74.112.64.75 <v-bebplbn_bbmgjmcoho_bfhnefea_bfhnefe...@bounce.sicirculation.mkt6348.com> to: sher...@ourdomain.com Message-Score: added 50 for DNSBL: failed, 74.112.64.75 listed in hostkarma.junkemailfilter.com, total score for this message is now 105; Jul-10-14 03:09:49 m-40497-02320 [SSL-out] [DNSBL] 74.112.64.75 <v-bebplbn_bbmgjmcoho_bfhnefea_bfhnefe...@bounce.sicirculation.mkt6348.com> to: sher...@ourdomain.com [scoring:50] (DNSBL: failed, 74.112.64.75 listed in (hostkarma.junkemailfilter.com<-127.0.1.1; )); In the second example, bl.spamcannibal.org has been assigned a weight of 35 as follows: bl.spamcannibal.org=>35 >From the log: Jul-10-14 10:22:15 m-40500-02646 [SSL-in] [SSL-out] 17.151.1.94 <boun...@insideicloud.icloud.com> to: lfar...@ourdomain.com Message-Score: added 50 for DNSBL: failed, 17.151.1.94 listed i n bl.spamcannibal.org, total score for this message is now 90; Jul-10-14 10:22:15 m-40500-02646 [SSL-in] [SSL-out] [DNSBL] 17.151.1.94 <boun...@insideicloud.icloud.com> to: lfar...@ourdomain.com [scoring:50] (DNSBL: failed, 17.151.1.94 listed in (bl.spamcannibal.org<-127.0.0.2; )); _____________________________________________ From: Phil Quesinberry Sent: Monday, July 07, 2014 1:08 PM To: 'assp-user@lists.sourceforge.net' Subject: RE: Inconsistent DNSBL scoring Ok... let me try to explain this a little better and then perhaps you can tell us why: one message gets a single hit on the ipews blacklist (and no others) with a weight of 20 and is scored with a value of 20 out of 50 corresponding to a Neutral DNSBL result with a resulting score of 30 added to the message score, then later the same day another message gets a single hit on the same ipews blacklist (and no others) with a weight of 20 but that one is scored with a value of 50 out of 50 corresponding to a Failed DNSBL result with a resulting score of 50 added to the message score? - Phil > From: Thomas Eckardt <Thomas.E<ckardt@th...> - 2014-07-07 15:52:48 >different results, possibly dependent upon whether it's a cached hit or not the difference is not cache or no cache - the important difference is 'neutral' and 'failed' - who can read .... - both have different valence values - 30 and 50 in your case IMHO using weights for DNSBL and URIBL is still working in V1 Thomas _____________________________________________ From: Phil Quesinberry Sent: Monday, July 07, 2014 11:41 AM To: 'assp-user@lists.sourceforge.net' Subject: RE: Inconsistent DNSBL scoring Looking at the log, you can see where a hit on the same blacklist yields two different results, possibly dependent upon whether it's a cached hit or not. This causes false positives when a more aggressive blacklist is intermittently and improperly scored the same way as say, Spamhaus. URIBL hits appear to exhibit the same behavior. I'm hoping that V2 is in better shape now than when we tried it last, which was admittedly quite some time ago. I'm not sure why this behavior was changed in V1 as it worked perfectly before. Looking back, I'm pretty sure that the inconsistent scoring behavior started after the "improvement". :) From: Thomas Eckardt <Thomas.E<ckardt@th...> - 2014-07-04 06:10:02 >added 30 for DNSBLcache: neutral >added 50 for DNSBL: failed >Now tell me what is wrong ? Also, is there a way to still have ASSP score the message based on total DNSBL points like it used to instead of limiting to the neutral/fail scoring dependent upon the number of hits? >use V2 >Thomas _____________________________________________ From: Phil Quesinberry Sent: Friday, July 04, 2014 12:12 AM To: 'assp-user@lists.sourceforge.net' Subject: RE: Inconsistent DNSBL scoring The same thing appears to be happening with URIBL hits as well. _____________________________________________ From: Phil Quesinberry Sent: Thursday, July 03, 2014 12:08 PM To: 'assp-user@lists.sourceforge.net' Subject: Inconsistent DNSBL scoring We're seeing some odd behavior when it comes to DNSBL scoring. One of the DNSBLs we use is i2.apews.org. It's useful for weighing in against a message with other problems but it's occasionally prone to false-positives, so we give hits against it a score of 20. Notice the following log entries (hit on apews DNSBL adds 20 which is the correct behavior): Jul-01-14 11:02:45 Connected: 108.12.183.50:31641 -> 10.0.0.15:25 (listenPort) -> 127.0.0.1:125; Jul-01-14 11:02:45 108.12.183.50 MTA offered STARTTLS - converting to SSL; Jul-01-14 11:02:47 m-40422-26873 [SSL-in] [SSL-out] 108.12.183.50 <brappap...@somesender.com> to: sher...@ourdomain.com Message-Score: added -5 for SSL-TLS-connection-OK, total score fo r this message is now -5; Jul-01-14 11:02:47 m-40422-26873 [SSL-in] [SSL-out] [DNSBL] 108.12.183.50 <brappap...@somesender.com> to: sher...@ourdomain.com [scoring:20] (108.12.183.50 listed in DNSBLcache by l2.ap ews.org at 2014-07-01/10:54:50); Jul-01-14 11:02:47 m-40422-26873 [SSL-in] [SSL-out] 108.12.183.50 <brappap...@somesender.com> to: sher...@ourdomain.com Message-Score: added 30 for DNSBLcache: neutral, 108.12.183.50 li sted in l2.apews.org, total score for this message is now 25; This is what we'd expect. Now have a look at the following log entries (hit on apews DNSBL adds 50 which is NOT correct), no configurational changes have been made: Jul-01-14 16:20:36 Connected: 173.67.34.179:54431 -> 10.0.0.15:25 (listenPort) -> 127.0.0.1:125; Jul-01-14 16:20:36 173.67.34.179 MTA offered STARTTLS - converting to SSL; Jul-01-14 16:20:38 m-40424-27323 [SSL-in] [SSL-out] 173.67.34.179 <d...@anothersender.com> to: ph...@ourotherdomain.com Message-Score: added -5 for SSL-TLS-connection-OK, total score for this message is now -5; Jul-01-14 16:20:38 m-40424-27323 [SSL-in] [SSL-out] 173.67.34.179 <d...@anothersender.com> to: ph...@ourotherdomain.com deleting spamming whitelisted tuplet: (173.67.34.0,anothersender.com) age: 2s; Jul-01-14 16:20:38 m-40424-27323 [SSL-in] [SSL-out] 173.67.34.179 <d...@anothersender.com> to: ph...@ourotherdomain.com Message-Score: added 50 for DNSBL: failed, 173.67.34.179 listed in l2.apews.org, total score for this message is now 45; Jul-01-14 16:20:38 m-40424-27323 [SSL-in] [SSL-out] [DNSBL] 173.67.34.179 <d...@anothersender.com> to: ph...@ourotherdomain.com [scoring:50] (DNSBL: failed, 173.67.34.179 listed in (l2.apews.org<-127.0.0.2; )); >From the above, It looks like cached hits are scored correctly but real-time hits are not? If you need additional info, let me know. I can crank up the DNSBL logging to debug/verbose if it will help. Also, is there a way to still have ASSP score the message based on total DNSBL points like it used to instead of limiting to the neutral/fail scoring dependent upon the number of hits? There are a number of DNSBLs which are useful for pushing suspect messages "over the edge" but more prone to false-positives and we'd prefer to have the granularity of just weighting them accordingly, which was the behavior of ASSP until fairly recently. Thanks! Phil Quesinberry Vintage Telecom VoIP Business Telephone Hosting Improve your business telephone services and save money (410) 921-6550 http://www.vintagetelecom.com ---------------------------------------------------------------------------- -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft _______________________________________________ Assp-user mailing list Assp-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-user DISCLAIMER: ******************************************************* This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! ******************************************************* ------------------------------------------------------------------------------ Want fast and easy access to all the code in your enterprise? Index and search up to 200,000 lines of code with a free copy of Black Duck Code Sight - the same software that powers the world's largest code search on Ohloh, the Black Duck Open Hub! Try it now. http://p.sf.net/sfu/bds _______________________________________________ Assp-user mailing list Assp-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-user
