Re: My new method for blocking spam - REVEALED!
--On Wednesday, January 20, 2016 4:26 PM -0500 Wrolf <wr...@wrolf.net> wrote: Is Marc's approach "novel" and "non-obvious"? (Patents must be novel, non-obvious, and useful.) I think plenty of people have supplied prior art, and that the concept itself is obvious since other things implement similar ideas. I.e,. see bogofilter, dspam, and bayes in general. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS lookups - bug with recursive lookups, or shoddy bind config?
--On Monday, January 04, 2016 8:28 PM + Chris J <c...@nightwolf.org.uk> wrote: Before I raise this on Bugzilla, I just want to run this past people as I'm quite happy that I've failed to configure something, but can't see what. In short, RBL blacklists haven't been working and I've finally, with tcpdump, traced it to SpamAssassin not requesting recursive queries. The setup is: Linux - Debian Jessie 8.2 Bind - 9.9.5-9+deb8u3-Debian SpamAssassin - installed from CPAN, 3.4.1 Perl - 5.20.2 Net::DNS - 1.01 If you're using Net::DNS 1.01 or later, you must patch SA. There is an entire thread dedicated to this issue. <https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7223> <https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7231> <https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7265> 7265 is only required for 1.03 (not necessary for 1.01, 1.02, or 1.04). --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS lookups fail with SpamAssassin since Net::DNS 1.03
--On Wednesday, December 16, 2015 4:13 PM + Ian Eiloart <i...@sussex.ac.uk> wrote: On 16 Dec 2015, at 16:09, Reindl Harald <h.rei...@thelounge.net> wrote: Am 16.12.2015 um 17:00 schrieb Ian Eiloart: On 16 Dec 2015, at 15:30, Kevin A. McGrail <kmcgr...@pccc.com> wrote: Downgrade tour netdns. There were changes in 1.03 that are fixed in trunk. Regards, KAM Downgrade? I upgraded to 1.04: does that not fix the problem? you answered that question at your own by only get SPF_NONE A fair point! Is anyone else seeing the same problem? As noted by Mark, there were changes for Net::DNS 1.01 and later that must be applied to SA 3.4.1 if you want it to work with 1.01 or later, completely irrespective of the Net::DNS 1.03 issues. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS lookups fail with SpamAssassin since Net::DNS 1.03
--On Wednesday, December 16, 2015 6:28 PM +0100 Mark Martinec <mark.martinec...@ijs.si> wrote: Tried it now with 3.4.1 and Net::DNS 1.04. You still need to apply the patch from Bug 7223 (in addition to a patch from Bug 7231), then it passes all tests with Net::DNS 1.04 (even without patches from Bug 7265). Seems easiest to install SpamAssassin from a svn 3.4 branch ( svn checkout http://svn.apache.org/repos/asf/spamassassin/branches/3.4 spamassassin-3.4 ) or downgrade Net::DNS to a pre-1.* version (i.e. 0.83). Hi Mark, I noticed that some of the changes for 7231 are only in trunk (DNS.pm, Plugin/AskDNS.pm), although those modules both exist in the 3.4 branch, and the changes are applicable. Is there any reason not to apply them if a version >= Net::DNS 0.69 will be being used? --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Net::DNS 1.0x should be avoided with SA 3.4.1
--On Wednesday, December 09, 2015 1:27 PM -0800 Quanah Gibson-Mount <qua...@zimbra.com> wrote: In testing in my lab, I've found significant issues using SpamAssassin 3.4.1 with Net::DNS 1.02 or later. Previously, I was using 0.81. This appears to be <https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7223> Will apply those revisions and retest. In general, it does appear to be the case that Net::DNS 1.0x should be entirely avoided with SA 3.4.1. ;) --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Net::DNS 1.0x should be avoided with SA 3.4.1
In testing in my lab, I've found significant issues using SpamAssassin 3.4.1 with Net::DNS 1.02 or later. Previously, I was using 0.81. With Net::DNS 1.02 or 1.04, there is an 15 second+ delay in delivering email. With debugging enabled for SA, we see the first delay here: Dec 9 15:19:56 zre-ldap002 amavis[14134]: (14134-03) p.path testus...@zre-ldap002.eng.zimbra.com: "P=p003,L=1,M=multipart/alternative | P=p002,L=1/2,M=text/html,T=asc" Dec 9 15:20:02 zre-ldap002 amavis[14134]: (14134-03) SA dbg: dns: select timed out 1.000 s Dec 9 15:20:02 zre-ldap002 amavis[14134]: (14134-03) SA dbg: async: select found no responses ready (t.o.=1.0) Dec 9 15:20:02 zre-ldap002 amavis[14134]: (14134-03) SA dbg: async: queries completed: 0, started: 0 This 6 second delay is consistent. It is followed by a ton of SA dbg: dns messages. Eventually we get to: Dec 9 15:20:11 zre-ldap002 amavis[14134]: (14134-03) SA dbg: async: escaping: lost or timed out requests or responses Dec 9 15:20:11 zre-ldap002 amavis[14134]: (14134-03) SA dbg: async: aborting after 15.010 s, past original deadline: TXT, askdns:TXT:_dmarc.zre-ldap002.eng.zimbra.com Dec 9 15:20:11 zre-ldap002 amavis[14134]: (14134-03) SA dbg: async: aborting after 15.003 s, past original deadline: NO_DNS_FOR_FROM, DNSBL-A, dns:A:zre-ldap002.eng.zimbra.com Dec 9 15:20:11 zre-ldap002 amavis[14134]: (14134-03) SA dbg: async: aborting after 15.002 s, past original deadline: NO_DNS_FOR_FROM, DNSBL-MX, dns:MX:zre-ldap002.eng.zimbra.com So this makes sense for 15 seconds, since that seems to be the timer default. I've even patched SA with the two commits from Mark to workaround the issues in 1.03 that were introduced. However, that doesn't change the problem I see here, so there are still apparently issues with using current versions of Net::DNS with SpamAssassin. With Net::DNS 0.81, I never see any dns debug lines from SA: Dec 9 14:48:45 zre-ldap002 amavis[428]: (00428-03) p.path testus...@zre-ldap002.eng.zimbra.com: "P=p003,L=1,M=multipart/alternative | P=p002,L=1/2,M=text/html,T=asc" Dec 9 14:48:45 zre-ldap002 postfix/amavisd/smtpd[3048]: 7F1021160815: client=localhost[127.0.0.1] Dec 9 14:48:45 zre-ldap002 postfix/cleanup[2268]: 7F1021160815: message-id=<1744051725.1.1449694124920.javamail.zim...@zre-ldap002.eng.zimbra.com> Dec 9 14:48:45 zre-ldap002 postfix/amavisd/smtpd[3048]: disconnect from localhost[127.0.0.1] ehlo=1 mail=11 rcpt=11 data=11 noop=1 quit=1 commands=36 Dec 9 14:48:45 zre-ldap002 postfix/qmgr[2267]: 7F1021160815: from=<testus...@zre-ldap002.eng.zimbra.com>, size=2457, nrcpt=1 (queue active) Dec 9 14:48:45 zre-ldap002 postfix/smtp[3010]: 35C101160816: to=<testus...@zre-ldap002.eng.zimbra.com>, relay=127.0.0.1[127.0.0.1]:10032, delay=0.34, delays=0.07/0/0/0.27, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 7F1021160815) Dec 9 14:48:45 zre-ldap002 postfix/qmgr[2267]: 35C101160816: removed Dec 9 14:48:45 zre-ldap002 postfix/lmtp[3087]: 7F1021160815: to=<testus...@zre-ldap002.eng.zimbra.com>, relay=zre-ldap002.eng.zimbra.com[10.137.242.52]:7025, delay=0.25, delays=0.03/0/0.1/0.12, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK) Instead, I get immediate delivery. Generally, I'd recommend against using current versions of Net::DNS until this can get sorted out. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS lookups fail with SpamAssassin since Net::DNS 1.03
--On Friday, November 13, 2015 2:01 PM -0500 "Kevin A. McGrail" <kmcgr...@pccc.com> wrote: On 11/13/2015 2:00 PM, Mark Martinec wrote: To me, this is an incompatible documented change - not something one would expect in an 1.02 -> 1.03 update. +1. An API change in a minor rev is not acceptable. Net::DNS 1.04 is out, fixing these issues. So far, it works better for me than 1.20 or 1.30 did in my lab. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS lookups fail with SpamAssassin since Net::DNS 1.03
--On Tuesday, December 08, 2015 4:55 PM -0800 Quanah Gibson-Mount <qua...@zimbra.com> wrote: --On Friday, November 13, 2015 2:01 PM -0500 "Kevin A. McGrail" <kmcgr...@pccc.com> wrote: On 11/13/2015 2:00 PM, Mark Martinec wrote: To me, this is an incompatible documented change - not something one would expect in an 1.02 -> 1.03 update. +1. An API change in a minor rev is not acceptable. Net::DNS 1.04 is out, fixing these issues. So far, it works better for me than 1.20 or 1.30 did in my lab. Err, 1.02 and 1.03. ;) --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS lookups fail with SpamAssassin since Net::DNS 1.03
--On Friday, November 13, 2015 10:22 AM -0800 Quanah Gibson-Mount <qua...@zimbra.com> wrote: Well, IO::Socket::IP support is new in Net::DNS 1.03, but it is only used if IO::Socket::INET6 is not present. I would assume you can use it as long as you have IO::Socket::INET6 installed, but I haven't tested that assumption. Although looking at the change log, it might not be specific to IO::Socket::IP: Fix rt.cpan.org #84375 Timeout doesn't work with bgsend/bgread Fix rt.cpan.org #47050 persistent sockets for Resolver::bg(send|read|isready) --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS lookups fail with SpamAssassin since Net::DNS 1.03
--On Friday, November 13, 2015 1:20 AM +0100 Mark Martinec <mark.martinec...@ijs.si> wrote: Net::DNS 1.03 breaks compatibility with SpamAssassin: DNS lookups no longer work, and warnings like the following pop up: lookup failed: Can't locate object method "handles" via package "IO::Socket::IP" at /usr/local/lib/perl5/site_perl/Net/DNS/Resolver/Base.pm line 735. There is a CPAN ticket open for this: https://rt.cpan.org/Public/Bug/Display.html?id=108745 Please stick to Net::DNS 1.02 until this is resolved. Well, IO::Socket::IP support is new in Net::DNS 1.03, but it is only used if IO::Socket::INET6 is not present. I would assume you can use it as long as you have IO::Socket::INET6 installed, but I haven't tested that assumption. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: effectiveness of DCC checks?
I just wanted to give a thank you to everyone who responded to this thread. I clearly misunderstood what DCC does, and it now has little value to me as a scoring item. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: effectiveness of DCC checks?
--On Tuesday, April 14, 2015 11:05 PM +0100 Steve Freegard s...@fsl.com wrote: On 14/04/15 19:45, Reindl Harald wrote: Am 14.04.2015 um 20:26 schrieb Kevin A. McGrail: On 4/14/2015 2:16 PM, Reindl Harald wrote: DCC isn't designed to tell you if a message is spam/not-spam. It's a *BULK* indicator. e.g. have lots of people seen this message? that is simply not true and defeats the purpose Yeah - but it's clear from other posting on this list that you'd argue black is in fact actually white. because i can't find any sense in give bulk mail just because it is bulk mail - indepdendent of subscribed, double-optin and what not - a penalty Just because *you* can't find any sense in it; others might be able to. For example: meta __FSL_ANY_BULK ((DCC_CHECK || RAZOR2_CHECK || PYZOR_CHECK) !FSL_EMPTY_BODY) meta FSL_FREEMAIL_BULK (__FSL_ANY_BULK FREEMAIL_FROM) scoreFSL_FREEMAIL_BULK 3.0 describe FSL_FREEMAIL_BULK Mail from Freemail account that matches bulk signature # 1.008 1.0844 0.1.000 0.750.00 + FSL_FREEMAIL_BULK However - I'll readily agree with you that DCC_CHECK adding score to all bulk mail isn't that useful, however that is what the mass-checker has decided works best with the corpus of mail available. Hi Steve, What is your rule for FSL_EMPTY_BODY? Your meta looks useful. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
effectiveness of DCC checks?
I've noticed that DCC_CHECK is flagging on tons of items that are clearly not spam. The most recent hit for me today was a release announcement from the mariadb folks. Overall, it's a trend I'm routinely seeing where it is flagging a lot of email that clearly isn't spam. Are others who use DCC seeing similar issues? --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Skipping RBL checks for internal servers
I noticed that some of the Zimbra auto-generated emails (reports on various bits) are getting hit with RBL scoring for some customers. This appears to be because they are (quite reasonably) using private IPs on some of thier internal Zimbra servers. However, when it goes through the MTA, it gets hit as spam because of this. Example: X-Spam-Status: Yes, score=10.297 tagged_above=-10 required=10 tests=[ALL_TRUSTED=-1, BAYES_00=-0.5, T_RP_MATCHES_RCVD=-0.01, URIBL_BLACK=3.25, URIBL_DBL_SPAM=2.5, URIBL_JP_SURBL=1.25, URIBL_RHS_DOB=1.514, URIBL_SBL_A=0.1, URIBL_WS_SURBL=1.608, URI_HEX=1.122, URI_NOVOWEL=0.5, URI_TRY_3LD=0.963, DSPAM.Innocent=-1.000] autolearn=no autolearn_force=no The originating IP is Received: from zcs1.example.com (LHLO zcs1.example.com) (10.2.0.3) The IP is clearly listed in trusted_networks, as can be seen via the ALL_TRUSTED scoring. Is there any way to write a rule that says if this came in via a trusted host, to skip RBL lookups? Or at least, specific servers? Thanks! --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Skipping RBL checks for internal servers
--On Wednesday, March 18, 2015 11:11 PM +0100 Reindl Harald h.rei...@thelounge.net wrote: The IP is clearly listed in trusted_networks your problem are not RBL's your problem are URIBL's and so mail content ask yourself why autogenerated mails contains crap URLs listed on URIBL_BLACK, URIBL_JP_SURBL *and* URIBL_WS_SURBL Well, it's a daily mail report... So it's listing a lot of information about who has connected, etc. So that makes sense that the content of it could contain blacklisted sites. I'll see if the client has configured SPF and/or DKIM, but based on the headers, I'd guess no. ;) Thanks for the pointer, that'll help immensely! --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: crm114 usage
--On Monday, March 09, 2015 11:04 PM +0100 Axb axb.li...@gmail.com wrote: On 03/09/2015 08:00 PM, Quanah Gibson-Mount wrote: Is anyone using crm114 still these days for scoring with in SpamAssassin? If so, does it seem to be an additional effective tool in helping to classify and score spam/ham? I used to use it with MailScanner about in the late 2000s but but as I couldn't find an efficient way to use it in a farm, I dropped it. Back then it seemed very complicated. There was a SA plugin for SA 3.x (http://mschuette.name/wp/crm114-spamassassin-plugin/) no idea if thats stufff is still mantained.. Ok, thanks. We suffer endless complaints about too much spam getting through SA, so I'm trying to find anything I can do to help improve scoring. ;) I'll be pushing out SA-trunk (3.4.1 beta) fairly soon, with the DMARC rules that were posted here. I'm hoping that helps significantly. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
crm114 usage
Is anyone using crm114 still these days for scoring with in SpamAssassin? If so, does it seem to be an additional effective tool in helping to classify and score spam/ham? Thanks! --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: unsubscribe
--On Wednesday, November 26, 2014 2:06 PM +0100 Axb axb.li...@gmail.com wrote: Girls, ^ - Extremely sexist. Please try some other form of insult in the future. ;) --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: DNS checks not being performed-
--On November 11, 2014 at 7:38:08 PM +0100 Reindl Harald h.rei...@thelounge.net wrote: What do you think I specifically installed in arandom location and in root's homedir? *you* wrote OK, looks like it's using /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/Socket.pm /root/.cpan/build/Socket-2.016-h4Od19/Socket.pm That's where it was built, not where it was installed. When you build via CPAN, it creates a .cpan in the ~user directory. So all this means is, they built Socket.pm using the cpan utility as the root user. It says nothing about where the resulting build was installed. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: Hacked sites: dropbox/googlebox/banking
--On November 3, 2014 at 7:52:10 AM -0800 John Hardin jhar...@impsec.org wrote: On Mon, 3 Nov 2014, Reindl Harald wrote: in fact we can kill them all by a single rule and so extend it to future filenames or foldernames uri RH_URI_MLW_ZEROHOUR /\/(dropbox|googlebox|banking)\/(document|doc|invoice)\.php$/ score RH_URI_MLW_ZEROHOUR 100 Adding a tuned version of this to my sandbox right now. Care to share the tuned version? --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: CYA .link
--On Tuesday, October 28, 2014 6:06 PM +0100 Axb axb.li...@gmail.com wrote: Patience quota exceeded. What a weird way to get a new TLD's ROI if (version = 3.004000) blacklist_uri_host link endif Testing this on my MTA's now... --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: CYA .link
--On Tuesday, October 28, 2014 4:16 PM -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: --On Tuesday, October 28, 2014 6:06 PM +0100 Axb axb.li...@gmail.com wrote: Patience quota exceeded. What a weird way to get a new TLD's ROI if (version = 3.004000) blacklist_uri_host link endif Testing this on my MTA's now... Doesn't seem to work. Oct 28 17:22:35 edge02 amavis[35776]: (35776-08) spam-tag, fallenrollmenti...@vdsc.100web-hostingplusonline.link - x...@zimbra.com, Yes, score=6.7 tagged_above=-10 required=3 tests=[BAYES_50=0.8, DCC_CHECK=3.5, RP_MATCHES_RCVD=-0.8, URIBL_BLACK=3.2] autolearn=no autolearn_force=no This is with the updated RegistrarBoundaries.pm file --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: .link TLD spammer haven?
--On Thursday, October 23, 2014 11:56 PM +0100 Martin Gregorie mar...@gregorie.org wrote: On Thu, 2014-10-23 at 17:20 +0200, Axb wrote: As there's a bunch of other new TLDs being abused I would higly recomend updating RegistrarBoundaries.pm from http://svn.apache.org/repos/asf/spamassassin/trunk/lib/Mail/SpamAssassin /Util/RegistrarBoundaries.pm on a Redhat flavour it goes in: locate RegistrarBoundaries.pm /usr/local/share/perl5/Mail/SpamAssassin/Util/RegistrarBoundaries.pm I updated this file yesterday. btw, the file includes instructions so you can update your own file without depending on a SA dev remembering to do it. Thanks for that. I've now installed it and have been running tests against my spam corpus to make sure that this subrule: uri __MG_LTD1 /\.link/i was now working correctly. Its hit all the stuff I thought it should, but my subrule turned out to be deficient because it will also hit any URI containing .linkedin, so anybody who has copied it should rewrite that rule so it looks like this: uri __MG_LTD1 /(\.link$|\.link\/)/i Even with that change, it always hits mail from linkedin --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: sanitizing/normalizing messages for feeding sa-learn
--On Wednesday, August 27, 2014 6:06 PM -0400 btb listsb-spamassas...@bitrate.net wrote: hi- we have a system [zimbra] where users can select a message in the mua interface and click a spam or not spam button. this generates a message [containing the selected message] which is ultimately delivered to a mailbox. i intend on retrieving these messages via imap and feeding sa-learn, but they've been a bit adulterated by the time they're retrieved, and i believe some cleanup is probably necessary prior to feeding sa-learn. That seems rather convoluted, given that Zimbra already trains its SA database automatically on a nightly basis based on the messages user submit via marking things as Spam. Are you running your own SA outside of Zimbra? --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Missing rules
--On Thursday, August 07, 2014 10:37 AM -0400 James B. Byrne byrn...@harte-lyne.ca wrote: On Wed, August 6, 2014 17:30, Quanah Gibson-Mount wrote: --On Wednesday, August 06, 2014 6:24 PM -0400 James B. Byrne byrn...@harte-lyne.ca wrote: I am constrained to run the version provided by the upstream distro packager (RedHat). When they update SA then, and only then, will I get the upgrade. Policies such as this show a complete lack of understanding on how to run production infrastructure. RH will never update SA in RHEL6 to any new release. Your best course of action is to fix your broken policy. Failing that, you can try finding a distribution that ships a newer build of SA, but whatever that is will quickly be outdated as well. Which explains, of course, why Linux distributions belonging to the RedHAt/CentOs/ScientificLinux/RHOS/ClearOS family are so lacking in popularity and so seldom found in corporate environments. Experienced admins understand the difference of having a base OS for their server, and actually using the god-awful horribly broken, incorrectly modified, vastly outdated, and generally destroyed packages they ship with the OS. RHEL6, for example, has an openldap build that's 4+ years old, and has an unsupported hack put into the RHEL build that missed a commit from years ago that protects against memory corruption. Debian/Ubuntu have done similar things (Remember the debian OpenSSH flaw some years back?). You use the outdated and questionably modified packages provided by distrubtions at extreme risk. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: rule for repeated tracking numbers
--On Wednesday, August 06, 2014 4:37 PM +0100 Paul Stead
paul.st...@zeninternet.co.uk wrote:
I've been having a play with the two rules mentioned, this seems to work
for me:
header __LOC_DIGITS_FROM From:name =~ /\.\d{7,8}$/
body __LOC_DIGITS_CONFUSER / (\d){7,8} .{1,250} ([0-9a-f]{32})
.{1,250}[\g1|\g2].{1,250}[\g1|\g2]/
Joining these together in a meta rule seems to be picking up the emails
I expect them to.
Would you be willing to share your full finalized ruleset? This spam is
really obnoxious.
Thanks!
--Quanah
--
Quanah Gibson-Mount
Server Architect
Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Re: rule for repeated tracking numbers
--On Wednesday, August 06, 2014 7:32 PM +0100 Paul Stead
paul.st...@zeninternet.co.uk wrote:
06/08/14 16:28, Quanah Gibson-Mount wrote:
Would you be willing to share your full finalized ruleset? This spam is
really obnoxious.
Sure...
A little adjustment as I noticed the brackets around the first number
match was wrong:
header __LOC_DIGITS_FROM From:name =~ /\.\d{7,8}$/
body __LOC_DIGITS_CONFUSER / (\d{7,8}) .{1,250} ([0-9a-f]{32})
.{1,250}[\g1|\g2] .{1,250}[\g1|\g2]/
Something like...
meta LOC_DIGITS_SPAM ( __LOC_DIGITS_FROM __LOC_DIGITS_CONFUSER)
score LOC_DIGITS_SPAM 0.001
should work
Thank you very much! I'm going to give it a test run on our server.
--Quanah
--
Quanah Gibson-Mount
Server Architect
Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Re: Missing rules
--On Wednesday, August 06, 2014 6:24 PM -0400 James B. Byrne byrn...@harte-lyne.ca wrote: I am constrained to run the version provided by the upstream distro packager (RedHat). When they update SA then, and only then, will I get the upgrade. Policies such as this show a complete lack of understanding on how to run production infrastructure. RH will never update SA in RHEL6 to any new release. Your best course of action is to fix your broken policy. Failing that, you can try finding a distribution that ships a newer build of SA, but whatever that is will quickly be outdated as well. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: More text/plain questions
--On Wednesday, July 23, 2014 9:39 PM +0100 Martin Gregorie
mar...@gregorie.org wrote:
On Wed, 2014-07-23 at 11:45 -0600, Amir 'CG' Caspi wrote:
I'm definitely considering writing a rule to catch #x0[0-9]{3};
patterns. I'm definitely worried it could cause FPs, but are there
common circumstances where legitimate emails would include dozens to
hundreds of these? (The latest FNs only include a few dozen, not the
hundreds seen in the spample above.)
This works for me:
describe MG_HEX_HTML Body contains too many HTML hex encodings
body MG_HEX_HTML /(.{0,3}\\#x[0-9A-F]{4};){5}/
scoreMG_HEX_HTML 3.5
It is also used in a meta, along with some other simple local rules, to
give hex-bearing spam an extra kick up the rear. I found that, in my
mailstream anyway, there was generally not much else to write rules
against, hence the high score. Spam arriving here gets quarantined: I
look at the sender and subject as a matter of course and, if it looks
like a possible FP, I'll look at the text too (I wrote a PHP viewer for
quarantined spam a long time ago) but it appears that, after the brief
squall of hex spam which made me write the rule, the promised spamstorm
ended and so far has failed to restart.
I've seen this rule hit several times for me today, all on definite spam.
--Quanah
--
Quanah Gibson-Mount
Server Architect
Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Re: colors TLDs in spam
--On Friday, August 01, 2014 4:14 PM +0200 Axb axb.li...@gmail.com wrote: On 08/01/2014 02:59 PM, Joe Quinn wrote: New TLDs are committed to trunk (revision 1615088). Thanks Joe! The process to update the TLDs is commented in RegistrarBoundaries.pm, so anyone is able to do it. Wanted to discuss the uper/lowercase re stuff before giving it a try ... I just got hit with pink: Return-Path: harassm...@famous.pink --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Wednesday, July 16, 2014 1:44 PM +0100 Martin Hepworth max...@gmail.com wrote: So whats the forwarder as it leaves your machine, a local DNS server, the appliance you think is in the way or Rackspace's DNS. If you can alter the overall forwarding so as it leaves your network can you make this google's or OpenDNS servers does this make a difference? dig @8.8.8.8 domain +trace results in the same behavior on the first lookup. So even bypassing our internal DNS servers doesn't alter the outcome. That'd throw out the IPv6 from Richard as well. In any case, our IT team now understands why this is an issue and is working to get it resolved ASAP. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Dealing with a bad network device affecting DNS lookups
Hi, Apparently there is a network device somewhere on the network my production servers use that is causing very long delays with first time DNS lookups. This is having a significant impact on SA's ability to score spam, as the various RBL lookups time out, as well as Razor and Pyzor. I've attempted to workaround this by setting: pyzor_timeout 60 razor_timeout 60 dcc_timeout 60 rbl_timeout 45 30 but I'm still seeing lookups being aborted. Here's an example of the problem: Jul 15 13:27:38 edge02 amavis[27683]: (27683-03) spam-tag, deg...@fullbaluster.co.uk - x...@zimbra.com, No, score=0.984 tagged_above=-10 required=3 tests=[BAYES_00=-0.05, DCC_CHECK=1.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RP_MATCHES_RCVD=-0.8, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01] autolearn=no autolearn_force=no Same email 2 seconds later, we can see Razor scoring is now there: Jul 15 13:28:40 edge02 amavis[27682]: (27682-06) spam-tag, deg...@fullbaluster.co.uk - x...@zimbra.com,a...@zimbra.com, Yes, score=6.413 tagged_above=-10 required=3 tests=[BAYES_00=-0.05, DCC_CHECK=1.1, DIGEST_MULTIPLE=0.293, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RAZOR2_CF_RANGE_51_100=0.5, RAZOR2_CF_RANGE_E8_51_100=1.886, RAZOR2_CHECK=2.75, RP_MATCHES_RCVD=-0.8, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01] autolearn=no autolearn_force=no So the second time it comes through, we get a valid spam tag. I most often see this with RBL lookups, which is a huge problem for scoring. Here's another example: First time run: X-Spam-Status: No, score=4.8 required=5.0 tests=DKIM_SIGNED, HTML_FONT_LOW_CONTRAST,HTML_IMAGE_RATIO_06,HTML_MESSAGE, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,RP_MATCHES_RCVD,T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.0 Second time run: X-Spam-Status: Yes, score=5.2 required=5.0 tests=DKIM_SIGNED, HTML_FONT_LOW_CONTRAST,HTML_IMAGE_RATIO_06,HTML_MESSAGE,NO_DNS_FOR_FROM, RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,RP_MATCHES_RCVD,T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.0 Note how NO_DNS_FOR_FROM is now added to the score set. In the successful run, I have: Jul 15 15:32:27.498 [52317] dbg: async: completed in 5.322 s: NO_DNS_FOR_FROM, DNSBL-MX, dns:MX:askpcm.com In the unsuccessful run, I have: Jul 15 15:28:14.563 [48690] dbg: async: aborting after 25.456 s, deadline shrunk: NO_DNS_FOR_FROM, DNSBL-MX, dns:MX:askpcm.com The next run, I have: Jul 15 15:32:27.498 [52317] dbg: async: completed in 5.322 s: NO_DNS_FOR_FROM, DNSBL-MX, dns:MX:askpcm.com So clearly my timeout values (45, 30) are not being honored, since 25 seconds 30 second minimum. Is there any way to set a global value of 60 seconds MINIMUM for all tests, period? Thanks! --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Tuesday, July 15, 2014 9:51 PM + Jeremy McSpadden jer...@fluxlabs.net wrote: Have you considered running your own DNS server locally ? I do. ;) But I don't run the network (our servers are hosted @ Rackspace), and any outbound DNS request hits the network appliance, so my own DNS doesn't help with this issue at all. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Tuesday, July 15, 2014 10:00 PM + Jeremy McSpadden jer...@fluxlabs.net wrote: Run a DNS server on your rack space servers. If your using rack space DNS your rbl queries are more than likely going to cause quite a few FPs. Never good to use ISP or hosting DNS servers. As I said... I *already* run my own DNS in rackspace. I *already* run my own caching nameserver too on my MTAs. That has ZERO to do with lookups against domains I don't host directly. I.e., *any* DNS request that goes through my DNS servers that then must go OUTBOUND hits the appliance on the rackspace network. 10.110.0.108 is *my* DNS server: For example, internal lookup (does not require going outbound): [quanah@mbs01 ~]$ time nslookup www.zimbra.com Server: 10.110.0.108 Address:10.110.0.108#53 Non-authoritative answer: www.zimbra.com canonical name = lb-www.zimbra.com. Name: lb-www.zimbra.com Address: 10.80.1.88 real0m0.011s user0m0.002s sys 0m0.009s External lookup (requires going outbound) 1st time: [quanah@mbs01 ~]$ time nslookup www.alltheweb.com ;; connection timed out; trying next origin Server: 10.110.0.108 Address:10.110.0.108#53 ** server can't find www.alltheweb.com: NXDOMAIN real0m18.008s user0m0.001s sys 0m0.004s External lookup (requires going outbound) 2nd time: [quanah@mbs01 ~]$ time nslookup www.alltheweb.com Server: 10.110.0.108 Address:10.110.0.108#53 Non-authoritative answer: www.alltheweb.com canonical name = rc.yahoo.com. rc.yahoo.comcanonical name = src.g03.yahoodns.net. src.g03.yahoodns.netcanonical name = any-src.a03.yahoodns.net. Name: any-src.a03.yahoodns.net Address: 74.6.50.150 real0m5.619s user0m0.004s sys 0m0.007s External lookup (requires going outbound) 3rd time: [quanah@mbs01 ~]$ time nslookup www.alltheweb.com Server: 10.110.0.108 Address:10.110.0.108#53 Non-authoritative answer: www.alltheweb.com canonical name = rc.yahoo.com. rc.yahoo.comcanonical name = src.g03.yahoodns.net. src.g03.yahoodns.netcanonical name = any-src.a03.yahoodns.net. Name: any-src.a03.yahoodns.net Address: 74.6.50.150 real0m0.011s user0m0.005s sys 0m0.005s --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Wednesday, July 16, 2014 12:08 AM +0200 Axb axb.li...@gmail.com wrote: and what's prevents you from running a recursor on those servers? In a halfway well connected network, and Rackpace is VERY well connected, DNS requests should takes less that 1 sec. The problem isn't the DNS requests. The problem is the appliance that is INTERCEPTING THE REQUESTS ON THE WAY OUT. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Tuesday, July 15, 2014 11:13 PM +0100 Martin Hepworth max...@gmail.com wrote: Run your own caching server on the sa box itself, makes a surprising difference and something I always reconmend *sigh* I DO already. That still does not prevent FIRST TIME LOOKUPS from failing. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Wednesday, July 16, 2014 12:12 AM +0200 Axb axb.li...@gmail.com wrote: And what appliance is that? No idea. Again, I don't run the network and what's on it. Whatever it is, if it breaks your DNS traffic, trash it. I have no control over it or its usage or presence. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Tuesday, July 15, 2014 3:41 PM -0700 John Hardin jhar...@impsec.org wrote: On Tue, 15 Jul 2014, Quanah Gibson-Mount wrote: --On Wednesday, July 16, 2014 12:12 AM +0200 Axb axb.li...@gmail.com wrote: And what appliance is that? No idea. Again, I don't run the network and what's on it. Whatever it is, if it breaks your DNS traffic, trash it. I have no control over it or its usage or presence. What does RackSpace IT say when you complain to them about this misbehavior? I've been complaining about it since last October. Supposedly it will be fixed by the end of this month. In the meantime, I still have floods of spam coming in that I'd like scored correctly. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Tuesday, July 15, 2014 3:52 PM -0700 Dave Warren da...@hireahit.com wrote: Are you saying that if you perform something like dig @8.8.8.8 asdfalksdflk.example.com a, Rackspace intercepts the packet on port 53 and does something with it? Right And it's taken them since October to resolve it? And you still pay for this service? Or is there more going on than is immediately obvious here? I honestly don't blame Rackspace for this specific problem. It has more to do with the environment as ordered by our IT department, and getting them to understand why the environment as-is is a problem, has been the difficulty. That has finally been done. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Dealing with a bad network device affecting DNS lookups
--On Wednesday, July 16, 2014 2:26 AM + lists-spamassassin inbound-lists-spamassas...@listmail.innovate.net wrote: I'm really not certain that using time and nslookup (which is a somewhat depreciated tool at this point) gives you results that show where the problem might be. I would suggest that for debugging/proof of issue purposes you use dig (which includes the query time by default) with options like +trace so that you can see what's really going on and how long it's taking at each stage of the lookup. You may have done this in the past, but the results output you included in this thread didn't do much to convince me that this was a Rackspace issue, rather than simply slow remote-end (e.g., yahoo) dns servers. It happens with *every* remote lookup the first time a domain is queried. It won't occur again for that domain until the cache expires on our local DNS. That was simply AN example of a domain I knew was likely to not be cached, since no one uses www.alltheweb.com anymore. ;) dig also returns NXDOMAIN on the first lookup. Here's a totally different domain, with dig: [quanah@mbs01 ~]$ time dig git-master.openldap.org +trace ; DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 git-master.openldap.org +trace ;; global options: +cmd . 339646 IN NS g.root-servers.net. . 339646 IN NS l.root-servers.net. . 339646 IN NS b.root-servers.net. . 339646 IN NS a.root-servers.net. . 339646 IN NS f.root-servers.net. . 339646 IN NS h.root-servers.net. . 339646 IN NS c.root-servers.net. . 339646 IN NS i.root-servers.net. . 339646 IN NS j.root-servers.net. . 339646 IN NS e.root-servers.net. . 339646 IN NS m.root-servers.net. . 339646 IN NS k.root-servers.net. . 339646 IN NS d.root-servers.net. ;; Received 508 bytes from 10.110.0.108#53(10.110.0.108) in 15 ms org.172800 IN NS a2.org.afilias-nst.info. org.172800 IN NS b0.org.afilias-nst.org. org.172800 IN NS b2.org.afilias-nst.org. org.172800 IN NS c0.org.afilias-nst.info. org.172800 IN NS a0.org.afilias-nst.info. org.172800 IN NS d0.org.afilias-nst.org. ;; Received 443 bytes from 192.5.5.241#53(192.5.5.241) in 15071 ms openldap.org. 86400 IN NS ns5.he.net. openldap.org. 86400 IN NS ns4.he.net. openldap.org. 86400 IN NS ns1.he.net. openldap.org. 86400 IN NS ns3.he.net. openldap.org. 86400 IN NS ns2.he.net. ;; Received 137 bytes from 199.19.53.1#53(199.19.53.1) in 10040 ms git-master.openldap.org. 300IN CNAME euler.openldap.org. euler.openldap.org. 300 IN A 23.92.27.229 ;; Received 77 bytes from 216.218.130.2#53(216.218.130.2) in 10 ms real0m27.152s user0m0.009s sys 0m0.020s --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
production MTA not doing URIBL lookups, why?
For some reason, my production MTA is not doing URIBL lookups for spam scoring, for no obvious reason. If I run a message through via the command line, I see the same behavior. If I run it through a test server, I see URIBL scores hit like mad. I do not appear to be blocked on my production MTA: [zimbra@edge01 ~]$ host -tTXT 2.0.0.127.multi.uribl.com 2.0.0.127.multi.uribl.com descriptive text permanent testpoint Message scoring for an obvious spam on prod gets: No, score=-0.8 required=5.0 tests=HTML_FONT_LOW_CONTRAST, HTML_IMAGE_RATIO_06,HTML_MESSAGE,RP_MATCHES_RCVD,T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.0 On my test server, I get: Yes, score=8.2 required=5.0 tests=DKIM_SIGNED, HTML_FONT_LOW_CONTRAST,HTML_IMAGE_RATIO_06,HTML_MESSAGE, RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,RCVD_IN_SBL, RP_MATCHES_RCVD,SPF_HELO_PASS,T_DKIM_INVALID,UNPARSEABLE_RELAY,URIBL_BLACK, URIBL_DBL_SPAM,URIBL_SBL,URIBL_SBL_A autolearn=no autolearn_force=no version=3.4.0 Obviously, I'd like my production server to be catching spam. ;) --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: production MTA not doing URIBL lookups, why?
--On Friday, July 11, 2014 4:44 PM -0700 John Hardin jhar...@impsec.org wrote: Prod also misses DKIM_SIGNED and SPF_HELO_PASS. Network tests disabled, maybe? Nope. Found the issue however. On my prod servers, I had the following set: dns_available test: 10.110.0.108 10.110.0.109 10.210.0.166 which are the IP addresses for my DNS servers. Unfortunately, with this line, SA always decides I don't have DNS for reasons that are beyond me, and then turns off the DNS checks. I've now changed it to: dns_available yes and things work as desired. So be very wary of telling SA to test DNS, because there's definitely something utterly broken there. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: production MTA not doing URIBL lookups, why?
--On Saturday, July 12, 2014 1:18 AM +0100 RW rwmailli...@googlemail.com wrote: Unfortunately, with this line, SA always decides I don't have DNS for reasons that are beyond me, It's clearly documented on the man page. Ah, yeah, I see that. I misread the first bit: By default, SpamAssassin will query some default hosts on the internet to attempt to check if DNS is working or not. as meaning that if I put in the test line, it'd change to querying the DNS servers I specified. :P --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
some questions on sa-compile
I'm looking at compiling the SA rules to get a measurement of the difference in SA timing to reduce delivery times for our email. I had a couple of questions first though: a) I assume that there's no issue uncommenting loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody even if I'm not using compiled rules. b) This statement is very vague, so I don't really know what the practical implications are: re2c can match strings much faster than perl code, by constructing a DFA to match many simple strings in parallel, and compiling that to native object code. Not all SpamAssassin rules are amenable to this conversion, however. Does this mean that: 1) The non-amenable rules are never processed? 2) The non-amenable rules are processed, but may be slower than if they weren't compiled? 3) ? Thanks! --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: some questions on sa-compile
--On May 2, 2014 at 3:47:22 PM -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Does this mean that: 1) The non-amenable rules are never processed? It more means they won't be compiled and you might not be able to compile them is more my understanding. I remember seeing the issue with sought rules where we couldn't compile them at which point I believe you run with nothing compiled. Perfect, thanks! --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
SA Info: dns: randomly showing up
Periodically, I'm finding dns: SA info lines coming from Amavis via SA. I'm not clear why these are triggered, and only for a few of the many thousands of emails processed per day. Insight appreciated, I'm running 3.4.0. An example: May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: no callback for id 31867/IN/A/206.139.235.66.zen.spamhaus.org, ignored; packet: ;; Answer received from 127.0.0.1 (124 bytes) May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; HEADER SECTION May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; id = 31867 May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; qr = 1 aa = 0 tc = 0 rd = 1 opcode = QUERY May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; ra = 1 z = 0 ad = 0 cd = 0 rcode = NXDOMAIN May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; qdcount = 1 ancount = 0 nscount = 1 arcount = 1 May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; do = 0 May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; EDNS version 0 May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; flags: May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; rcode: NOERROR May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; size: 4096 May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; option: May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; QUESTION SECTION (1 record) May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; 206.139.235.66.zen.spamhaus.org. IN A May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; ANSWER SECTION (0 records) May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; AUTHORITY SECTION (1 record) May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] zen.spamhaus.org. 150 IN SOA need.to.know.only. hostmaster.spamhaus.org. ( May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] 1405010901 ;serial May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] 3600 ;refresh May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] 600 ;retry May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] 432000 ;expire May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] 150 ) ;minimum May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; ADDITIONAL SECTION (1 record) May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; EDNS version 0 May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; flags: May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; rcode: NOERROR May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; size: 4096 May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: [...] ;; option: May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: no likely matching queries for id 31867 Thanks, Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SA Info: dns: randomly showing up
--On May 1, 2014 at 3:02:50 PM -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: On 5/1/2014 2:33 PM, Quanah Gibson-Mount wrote: Periodically, I'm finding dns: SA info lines coming from Amavis via SA. I'm not clear why these are triggered, and only for a few of the many thousands of emails processed per day. Insight appreciated, I'm running 3.4.0. An example: Are they all for spamhaus? Nope. May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: no callback for id 31867/IN/A/206.139.235.66.zen.spamhaus.org, ignored; packet: ;; Answer received from 127.0.0.1 (124 bytes) May 1 08:15:16 edge01 amavis[33725]: (33725-07) SA info: dns: no callback for id 15337/IN/A/zimbra.com.dob.sibl.support-intelligence.net, ignored; packet: ;; Answer received from 127.0.0.1 (140 bytes) May 1 09:16:09 edge01 amavis[14845]: (14845-03) SA info: dns: no callback for id 29158/IN/A/linkedin.com.dbl.spamhaus.org, ignored; packet: ;; Answer received from 127.0.0.1 (122 bytes) May 1 10:58:56 edge01 amavis[19422]: (19422-03) SA info: dns: no callback for id 42530/IN/A/zimbra.com.dob.sibl.support-intelligence.net, ignored; packet: ;; Answer received from 127.0.0.1 (140 bytes) May 1 11:03:12 edge02 amavis[33893]: (33893-19) SA info: dns: no callback for id 11196/IN/A/ns10.bac.com, ignored; packet: ;; Answer received from 127.0.0.1 (243 bytes) What version of Net::DNS are you running? $VERSION = '0.74'; Are you using locally cached DNS? Yes. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SA Info: dns: randomly showing up
--On May 1, 2014 at 3:45:00 PM -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Do you have any dns_options set in your configuration? This does seem to imply a DNS issue. Not currently: [zimbra@edge01 ~]$ cd conf/sa [zimbra@edge01 sa]$ ls salocal.cf sauser.cf [zimbra@edge01 sa]$ grep dns * [zimbra@edge01 sa]$ --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SA Info: dns: randomly showing up
--On May 2, 2014 at 3:09:37 AM +0200 Mark Martinec mark.martinec...@ijs.si wrote: Quanah Gibson-Mount wrote: Periodically, I'm finding dns: SA info lines coming from Amavis via SA. I'm not clear why these are triggered, and only for a few of the many thousands of emails processed per day. Insight appreciated, I'm running 3.4.0. An example: May 1 04:13:15 edge01 amavis[61006]: (61006-06) SA info: dns: no callback for id 31867/IN/A/206.139.235.66.zen.spamhaus.org, ignored; packet: ;; Answer received from 127.0.0.1 (124 bytes) https://rt.cpan.org/Public/Bug/Display.html?id=83451 Thanks! I see your suggested fix was rejected. :/ My takeaway from the comment is that SA needs to handle the alarms better when it comes to Net::DNS? --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
No URIDNSBL scanning?
-- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: No URIDNSBL scanning?
--On Monday, March 24, 2014 12:28 PM -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: For some reason, with this spam email, URIDNSBL never seems to kick off. Usually I see lines like: Ah, I didn't have the full text of the message. However, something still seems off, as the URIDNSBL scans aborted? Mar 24 13:33:51.109 [17508] dbg: uridnsbl: considering host=www.writicized.eu, domain=writicized.eu Mar 24 13:33:51.130 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_PH_SURBL DNSBL:writicized.eu:multi.surbl.org Mar 24 13:33:51.130 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_MW_SURBL DNSBL:writicized.eu:multi.surbl.org Mar 24 13:33:51.130 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SC_SURBL DNSBL:writicized.eu:multi.surbl.org Mar 24 13:33:51.130 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_JP_SURBL DNSBL:writicized.eu:multi.surbl.org Mar 24 13:33:51.130 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_AB_SURBL DNSBL:writicized.eu:multi.surbl.org Mar 24 13:33:51.130 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_WS_SURBL DNSBL:writicized.eu:multi.surbl.org Mar 24 13:33:51.130 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_RHS_DOB DNSBL:writicized.eu:dob.sibl.support-intelligence.net Mar 24 13:33:51.131 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_DBL_ERROR DNSBL:writicized.eu:dbl.spamhaus.org Mar 24 13:33:51.131 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_DBL_SPAM DNSBL:writicized.eu:dbl.spamhaus.org Mar 24 13:33:51.131 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_DBL_REDIR DNSBL:writicized.eu:dbl.spamhaus.org Mar 24 13:33:51.133 [17508] dbg: uridnsbl: complete_ns_lookup NS:writicized.eu Mar 24 13:33:51.135 [17508] dbg: uridnsbl: got(1) NS for writicized.eu: writicized.eu. 85335 IN NS b.ns.joker.com. Mar 24 13:33:51.135 [17508] dbg: uridnsbl: got(2) NS for writicized.eu: writicized.eu. 85335 IN NS a.ns.joker.com. Mar 24 13:33:51.136 [17508] dbg: uridnsbl: got(3) NS for writicized.eu: writicized.eu. 85335 IN NS c.ns.joker.com. Mar 24 13:33:51.137 [17508] dbg: uridnsbl: complete_a_lookup A:www.writicized.eu Mar 24 13:33:51.137 [17508] dbg: uridnsbl: complete_a_lookup got(1) A for www.writicized.eu: www.writicized.eu. 736 IN A 184.22.111.14 Mar 24 13:33:51.141 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL DNSBL:14.111.22.184:zen.spamhaus.org Mar 24 13:33:51.144 [17508] dbg: uridnsbl: complete_a_lookup A:b.ns.joker.com Mar 24 13:33:51.144 [17508] dbg: uridnsbl: complete_a_lookup got(1) A for b.ns.joker.com: b.ns.joker.com. 144125 IN A 159.25.97.69 Mar 24 13:33:51.146 [17508] dbg: uridnsbl: complete_a_lookup A:a.ns.joker.com Mar 24 13:33:51.146 [17508] dbg: uridnsbl: complete_a_lookup got(1) A for a.ns.joker.com: a.ns.joker.com. 144125 IN A 184.172.157.218 Mar 24 13:33:51.147 [17508] dbg: uridnsbl: complete_a_lookup A:c.ns.joker.com Mar 24 13:33:51.148 [17508] dbg: uridnsbl: complete_a_lookup got(1) A for c.ns.joker.com: c.ns.joker.com. 144125 IN A 85.25.110.247 Mar 24 13:33:51.149 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL_A DNSBL:14.111.22.184:sbl.spamhaus.org Mar 24 13:33:51.149 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL_A DNSBL:69.97.25.159:sbl.spamhaus.org Mar 24 13:33:51.150 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL DNSBL:69.97.25.159:zen.spamhaus.org Mar 24 13:33:51.150 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL_A DNSBL:218.157.172.184:sbl.spamhaus.org Mar 24 13:33:51.151 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL DNSBL:218.157.172.184:zen.spamhaus.org Mar 24 13:33:51.151 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL_A DNSBL:247.110.25.85:sbl.spamhaus.org Mar 24 13:33:51.152 [17508] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL DNSBL:247.110.25.85:zen.spamhaus.org Mar 24 13:33:54.879 [17508] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_BLACK DNSBL:writicized.eu:multi.uribl.com Mar 24 13:33:54.879 [17508] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_RED DNSBL:writicized.eu:multi.uribl.com Mar 24 13:33:54.879 [17508] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_GREY DNSBL:writicized.eu:multi.uribl.com Mar 24 13:33:54.879 [17508] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_BLOCKED DNSBL:writicized.eu:multi.uribl.com Return-Path: lendingt...@writicized.eu X-Spam-Status: No, score=2.4 required=5.0 tests=AC_HTML_NONSENSE_TAGS, HTML_FONT_LOW_CONTRAST,HTML_MESSAGE,MIME_HTML_ONLY,RDNS_NONE,TVD_RCVD_IP, TVD_RCVD_IP4,T_REMOTE_IMAGE,UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.0 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on edge01.zimbra.com --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Pyzor errors block URIBL lookups?
--On Thursday, March 13, 2014 3:50 PM -0700 John Hardin jhar...@impsec.org wrote: I've moved the discussion over to amavis-users@. It is very clear that current version of Amavis are utterly broken in handling SpamAssassin. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Pyzor errors block URIBL lookups?
In looking at why some spam is still making it through, it appears that Pyzor errors block URIBL lookups: Mar 13 13:15:23.849 [28433] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL_A DNSBL:1.193.124.98:sbl.spamhaus.org Mar 13 13:15:23.849 [28433] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL DNSBL:1.193.124.98:zen.spamhaus.org Mar 13 13:15:23.850 [28433] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL_A DNSBL:82.172.235.213:sbl.spamhaus.org Mar 13 13:15:23.959 [28433] dbg: uridnsbl: complete_dnsbl_lookup URIBL_RHS_DOB DNSBL:macrotermbed.com:dob.sibl.support-intelligence.net Mar 13 13:15:24.620 [28433] warn: pyzor: check failed: internal error, python traceback seen in response Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_BLACK DNSBL:macrotermbed.com:multi.uribl.com Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_RED DNSBL:macrotermbed.com:multi.uribl.com Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_GREY DNSBL:macrotermbed.com:multi.uribl.com Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_BLOCKED DNSBL:macrotermbed.com:multi.uribl.com Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_DBL_ERROR DNSBL:macrotermbed.com:dbl.spamhaus.org Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_DBL_SPAM DNSBL:macrotermbed.com:dbl.spamhaus.org Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_DBL_REDIR DNSBL:macrotermbed.com:dbl.spamhaus.org Mar 13 13:15:28.679 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_PH_SURBL DNSBL:macrotermbed.com:multi.surbl.org Mar 13 13:15:28.680 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_MW_SURBL DNSBL:macrotermbed.com:multi.surbl.org Mar 13 13:15:28.680 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_SC_SURBL DNSBL:macrotermbed.com:multi.surbl.org Mar 13 13:15:28.680 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_JP_SURBL DNSBL:macrotermbed.com:multi.surbl.org Mar 13 13:15:28.680 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_AB_SURBL DNSBL:macrotermbed.com:multi.surbl.org Mar 13 13:15:28.680 [28433] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_WS_SURBL DNSBL:macrotermbed.com:multi.surbl.org This seems like a bug to me. I would expect URIBL lookups to continue, regardless of the error from python so that proper scoring can be achieved. Is there anyway to disable this behavior? Should I open a bug? Version is SA 3.4.0 Thanks, Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Pyzor errors block URIBL lookups?
--On Thursday, March 13, 2014 1:48 PM -0700 John Hardin jhar...@impsec.org wrote: On Thu, 13 Mar 2014, Quanah Gibson-Mount wrote: In looking at why some spam is still making it through, it appears that Pyzor errors block URIBL lookups: I'm working with someone who seems to be having the same problem in 3.3.1 - thanks for noting this, I will take a closer look. Thanks. The scoring can vary depending on when the pyzor callback fails. For example, in another run, the pyzor error doesn't come back until after more of the URI checks are done: Mar 13 15:40:12.090 [6070] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL DNSBL:1.192.124.98:zen.spamhaus.org Mar 13 15:40:12.091 [6070] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL_A DNSBL:1.193.124.98:sbl.spamhaus.org Mar 13 15:40:12.091 [6070] dbg: uridnsbl: complete_dnsbl_lookup URIBL_SBL DNSBL:1.193.124.98:zen.spamhaus.org Mar 13 15:40:12.843 [6070] warn: pyzor: check failed: internal error, python traceback seen in response Mar 13 15:40:15.683 [6070] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_BLACK DNSBL:macrotermbed.com:multi.uribl.com Mar 13 15:40:15.683 [6070] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_RED DNSBL:macrotermbed.com:multi.uribl.com Mar 13 15:40:15.683 [6070] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_GREY DNSBL:macrotermbed.com:multi.uribl.com Mar 13 15:40:15.683 [6070] dbg: uridnsbl: complete_dnsbl_lookup aborted URIBL_BLOCKED DNSBL:macrotermbed.com:multi.uribl.com So I get a much higher score: Yes, score=10.1 required=5.0 Run it again, it fails earlier, and I get: Yes, score=8.1 required=5.0 Run it again, it fails later, and I get: Yes, score=12.8 required=5.0 etc. I.e., the scoring is completely erratic based on where URIBL processing is when the pyzor callback fails. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Pyzor errors block URIBL lookups?
--On Thursday, March 13, 2014 3:15 PM -0700 John Hardin jhar...@impsec.org wrote: FWIW they're running amavisd-new, and we're trying to figure out why the scores on MTA-processed messages are so much lower than when the same message is passed through command-line SA in debug mode. Hi John, Interesting -- I'm also running under amavisd-new (2.8.2 RC1). --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Pyzor errors block URIBL lookups?
--On Thursday, March 13, 2014 3:50 PM -0700 John Hardin jhar...@impsec.org wrote: On Fri, 14 Mar 2014, Jason Haar wrote: Just yesterday I manually pushed a piece of spam through spamc and spamassassin and got a different score too. It ended up being caused by time_limit. spamassassin didn't listen to it whereas spamc/spamd did and the email took a lng time to process - triggering the scores to be different I ended up just increasing time_limit to fix. In the amavisd config? Hm... I'm seeing really random scores across the board, pyzor or not (I commented out the die() so tha t cannot be the cause). For example: Mar 13 17:02:24 edge01 amavis[60025]: (60025-04) spam-tag, a...@mydomain.com - d...@mydomain.com,g...@mydomain.com, No, score=-1.149 tagged_above=-10 required=3 tests=[ALL_TRUSTED=-1, BAYES_00=-0.05, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001] autolearn=ham autolearn_force=no This is missing RCVD_IN_DNSWL_HI. This email originated on our MTAs, and was delivered going through them. This did too, and correclty has that rule applied: Mar 13 17:00:08 edge02 amavis[39369]: (39369-17-3) spam-tag, a...@mydomain.com - d...@zimbra.com, No, score=-5.149 tagged_above=-10 required=3 tests=[BAYES_00=-0.05, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, UNPARSEABLE_RELAY=0.001] autolearn=unavailable autolearn_force=no The difference is the timing... The second one came in at 17:00:07 and was marked scanned through SA by 17:00:08. The second one came in at 17:02:20, so there was a 4 second processing time in Amavis. The odd thing is that the amavisd default for child process timeouts is 8 minutes. The SA timeout for RBL lookups is 5 seconds. So my deliveries are well within those timeout boundaries. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Pyzor errors block URIBL lookups?
--On Thursday, March 13, 2014 4:27 PM -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: This is missing RCVD_IN_DNSWL_HI. This email originated on our MTAs, and was delivered going through them. This did too, and correclty has that rule applied: Mar 13 17:00:08 edge02 amavis[39369]: (39369-17-3) spam-tag, a...@mydomain.com - d...@zimbra.com, No, score=-5.149 tagged_above=-10 required=3 tests=[BAYES_00=-0.05, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, UNPARSEABLE_RELAY=0.001] autolearn=unavailable autolearn_force=no And here is another email, from the *same* user to the *same* user that does not have RCVD_IN_DNSWL_HI!!! Mar 13 19:21:07 edge02 amavis[3918]: (03918-12) spam-tag, a...@zimbra.com - d...@zimbra.com, No, score=-0.148 tagged_above=-10 required=3 tests=[BAYES_00=-0.05, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, UNPARSEABLE_RELAY=0.001] autolearn=ham autolearn_force=no Something is really, really wrong here. I'm guessing Amavis is the culprit. :/ --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Pyzor errors block URIBL lookups?
--On Thursday, March 13, 2014 6:25 PM -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: And here is another email, from the *same* user to the *same* user that does not have RCVD_IN_DNSWL_HI!!! Mar 13 19:21:07 edge02 amavis[3918]: (03918-12) spam-tag, a...@zimbra.com - d...@zimbra.com, No, score=-0.148 tagged_above=-10 required=3 tests=[BAYES_00=-0.05, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, UNPARSEABLE_RELAY=0.001] autolearn=ham autolearn_force=no Something is really, really wrong here. I'm guessing Amavis is the culprit. :/ Difference this time vs last time is that it took 3 seconds for Amavis to tag it, vs 1 second or less last time. It definitely sounds like Amavis has a very low timeout somewhere that is aborting spam checks. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Pyzor errors block URIBL lookups?
--On Friday, March 14, 2014 2:28 PM +1300 Jason Haar jason_h...@trimble.com wrote: No - I don't use amavis. That's why I said spamc :-) Well... The docs say time_limit defaults to 300 seconds (5 minutes). The inconsistent scoring I'm seeing is all occuring under 5 seconds, so I don't think it's related. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Status of v3.4
--On Tuesday, January 07, 2014 4:08 PM -0500 Alex mysqlstud...@gmail.com wrote: HI guys, I wanted to ask what the current status of v3.4 is since the beta was posted some months ago? Are people finding that it's already performing better than v3.2? Are there rule updates/improvements as frequently as with v3.2? Is it available as a tarball or should I just check it out with svn? We have been using 3.4 (various snapshots from svn) for over a year now, and it is (for us) a substantial improvement over 3.2. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 11:09 PM +0200 Axb axb.li...@gmail.com wrote: sent reply directly, sotrry - here's for the list On 10/22/2013 10:33 PM, Quanah Gibson-Mount wrote: I don't get the concern about VMW. The vmw hosts are *my* MTAs and in mynetworks. mail.zimbra.com - load balanced name for edge01-zcs.vmware.com, edge02-zcs.vmware.com The SPAM did not originate with my servers... It originated elsewhere. This is rather clear: Received: from c115-smtp.pumpery.com (c115-smtp.pumpery.com [5.135.12.243]) by edge02-zcs.vmware.com (Postfix) with ESMTP id 76999784 for ; Tue, 22 Oct 2013 11:27:05 -0700 (PDT) pumpery.com is the originator of this spam. I've blacklisted the from in the meantime. If pumpery.com was in the msg's body, the URIBL plugin should have detected them yet another snowshoer on OVH (5.135.12.128/25) I hope, for your health, that you're going to blacklist every from in a missed spam pumpery.com listed on black.uribl.com pumpery.com listed on jp.surbl.org pumpery.com listed on sc.surbl.org pumpery.com listed on dbl.spamhaus.org You've missed the point. mynetworks is not SA - it's Postfix and SA knows nothing about this config option. as you have SA configured, RBL lookups are done against the vmware IPs and I doubt those will be blacklisted, anywhere. So I've already confirmed this is *not* the case. My trusted_networks is correct as configured -- Yet spam that should be blacklisted by the RBLs continues to flow in. So, how do I determine why SA is failing to correctly query the RBLs? --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 8:00 PM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-10-23 19:37: pumpery.com listed on black.uribl.com pumpery.com listed on jp.surbl.org pumpery.com listed on sc.surbl.org pumpery.com listed on dbl.spamhaus.org this is urlbl, nothing to do with trusted_networks Axb's point was that if trusted_networks is not configured correctly, SA will not do the URLBL checks correctly. I'm noting that trusted_networks *is* configured correctly, and SA still does not appear to be doing the checks correctly since emails with blacklisted web links are still flooding my servers with spam. I.e., there is no score anywhere from these blacklists being added into my spam scores. The module is loaded: init.pre:loadplugin Mail::SpamAssassin::Plugin::URIDNSBL So, for example, I believe I should have seen a score for URIBL_DBL_SPAM, since the pumpery.com site is listed on dbl.spamhaus.org, and there were multiple HTML links in the email for pumpery.com in the email. 50_scores.cf:score URIBL_DBL_SPAM 0 1.7 0 1.7 So, how do I determine why SA is failing to correctly query the RBLs? rndc querylog if you have own bind9 running on localhost bind9 is not installed on localhost. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 3:58 PM -0400 Kris Deugau kdeu...@vianet.ca wrote: Only select headers have URIs extracted and passed to the DNS lookups; I don't *think* Received: or Message-Id: are included. I've been surprised now and then discovering a URI that *was* extracted from a header. Otherwise all URI lookups are done on URIs found in the message body. Ok, but the message body specifically has multiple links to pumpery.com. So why didn't it get scored? That's what I don't understand. ;) --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 10:06 PM +0200 Benny Pedersen m...@junc.eu wrote: if you have own bind9 running on localhost bind9 is not installed on localhost. so resolv.conf is forwarding in wild ? :( resolve.conf uses VMWare's DNS servers which are not located on the MX servers. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 10:34 PM +0200 Axb axb.li...@gmail.com
wrote:
pls grep your logs for one of these: URIBL , SURBL , DBL (uppercase)
Do you see any hits at all?
I see one:
Oct 23 09:05:53 edge01-zcs amavis[3250]: (03250-12) Checking: 4vlUublDBL_R
[162.213.112.166] x...@in.telligent.com - x...@zimbra.com
Oct 23 09:05:53 edge01-zcs amavis[3250]: (03250-12) Passed CLEAN
{RelayedInbound}, [162.213.112.166]:49611 [162.213.112.166]
x...@in.telligent.com - x...@zimbra.com, Queue-ID: A39DD79F, Message-ID:
dalopszfsp1j106zppp1...@dalops.corp.telligent.com, mail_id:
4vlUublDBL_R, Hits: -97.305, size: 7199, queued_as: 7ACA71295, 484 ms
It is BCP to use a local resolver under your control for mail servers.
Due to hammering public mirrors, an ISP/ASP's shared resolver may be
tarpitted or blocked from doing queries to the BLs.
If you run your own, you know when and what is happening and makes it
easier to troubleshoot /monitor any potential issues.
Yeah, it's on my to-do list to add local dnscaching software to the Zimbra
product. ;)
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 10:57 PM +0200 Benny Pedersen m...@junc.eu
wrote:
Quanah Gibson-Mount skrev den 2013-10-23 22:45:
Oct 23 09:05:53 edge01-zcs amavis[3250]: (03250-12) Checking:
4vlUublDBL_R [162.213.112.166] x...@in.telligent.com -
x...@zimbra.com
Oct 23 09:05:53 edge01-zcs amavis[3250]: (03250-12) Passed CLEAN
{RelayedInbound}, [162.213.112.166]:49611 [162.213.112.166]
x...@in.telligent.com - x...@zimbra.com, Queue-ID: A39DD79F,
Message-ID: dalopszfsp1j106zppp1...@dalops.corp.telligent.com,
mail_id: 4vlUublDBL_R, Hits: -97.305, size: 7199, queued_as:
7ACA71295, 484 ms
where is uribl hits here ?
It's the only instance of DBL anywhere, is all. ;) No other hits for the
strings.
is this mail gets -100 somewhere ?, too much whitelistning to not see the
problem ?
in.telligent.com is our parent company, so yes, we whitelist anything they
send.
http://blog.zimbra.com/blog/archives/2013/07/telligent-acquires-zimbra-from-vmware.html
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 10:52 PM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-10-23 22:09: Ok, but the message body specifically has multiple links to pumpery.com. So why didn't it get scored? That's what I don't understand. ;) X-ASF-Spam-Status: No, hits=7.1 required=10.0 tests=SPF_PASS,URIBL_BLACK,URIBL_DBL_SPAM,URIBL_JP_SURBL,URIBL_SC_SURBL error exists in localhost :=) Right, but where is the error? ;) That's the whole question. ;) In reading over http://wiki.apache.org/spamassassin/DnsBlocklists I came across this statement: A: Third, if your email gateway is behind a firewall make sure that SpamAssassin is resolving the gateway to its external address. If SpamAssassin resolves the gateway to an private IP or can't resolve the name at all, it may mark the sending system as a trusted relay. As a result, some or all of the spammer's systems will not be checked against the DNSBL. (I'm not aware of anyway to specify 'last trusted relay' in SA). and I wonder if that is the problem. The DNS that is used definitely resolves the MX to its internal IP, and not its external IP. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 5:04 PM -0400 Kris Deugau kdeu...@vianet.ca wrote: g Well, you didn't post the message body... *Usually* that indicates that the URI wasn't listed when the message was originally processed, but checking again even 10-15 minutes later it is. This is tricky to confirm unless you have enough access to the raw URI lists to know when the URI was added. Ok, that makes sense. ;) Post a complete example on pastebin - maybe there was something odd in the message structure that caused the URIs to be skipped, but I can't say I've ever seen one. SA goes to great lengths to mimic the idiocy that many mail clients go to in picking URIs out of the message. Bad grammar/typing with something like ... for dinner.It was ... is enough to cause dinner.it to get looked up, so it's much more likely the URI simply wasn't listed when the message was first scanned. http://ur1.ca/fxhkp Run the complete message through spamassassin -D uridnsbl message - you should get a line like: Oct 23 16:57:24.845 [12772] dbg: uridnsbl: domains to query: (hopefully with a list of URIs to actually query) Yeah, it definitely appears it is querying them correctly. The updated header even has: X-Spam-Checker-Version: SpamAssassin 3.4.0-pre3-r1435395 (2013-01-18) on edge02-zcs.vmware.com X-Spam-Level: ** X-Spam-Status: No, score=2.3 required=5.0 tests=DKIM_SIGNED, HTML_IMAGE_RATIO_02,HTML_MESSAGE,RP_MATCHES_RCVD,T_DKIM_INVALID, T_HEADER_FROM_DIFFERENT_DOMAINS,UNPARSEABLE_RELAY,URIBL_BLOCKED, URIBL_DBL_SPAM autolearn=no version=3.4.0-pre3-r1435395 Among the other bits, handy things like: Oct 23 14:18:43.636 [24474] dbg: uridnsbl: domain pumpery.com listed (URIBL_BLOCKED): 127.0.0.1 Oct 23 14:18:43.638 [24474] dbg: uridnsbl: domain pumpery.com listed (URIBL_DBL_SPAM): 127.0.1.2 Oct 23 14:18:43.739 [24474] dbg: uridnsbl: domain nsports.com.br listed (URIBL_BLOCKED): 127.0.0.1 So I guess it wasn't listed at the time the message came in, as you noted. Still the spam score seems a bit low, I guess I may want to tweak the URIBL_DBL_SPAM and URIBL_BLOCKED scores. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 11:32 PM +0200 Axb axb.li...@gmail.com wrote: URIBL_BLOCKED is not good news .-) I wouldn't touch that score... http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block Ok. Here is something I don't understand -- Why I get utterly different values from email that goes through the MTA, and the SA command line. I *just* received an email, with the following scoring: X-Spam-Flag: NO X-Spam-Score: 2.717 X-Spam-Level: ** X-Spam-Status: No, score=2.717 tagged_above=-10 required=3 tests=[BAYES_50=0.8, HTML_EXTRA_CLOSE=0.001, HTML_MESSAGE=0.001, RDNS_NONE=0.793, URI_HEX=1.122] autolearn=no So I dumped it to a text file, and ran it through SA from the command line, and I get: X-Spam-Checker-Version: SpamAssassin 3.4.0-pre3-r1435395 (2013-01-18) on edge02-zcs.vmware.com X-Spam-Level: *** X-Spam-Status: No, score=4.0 required=5.0 tests=RCVD_IN_MSPIKE_H2,RCVD_IN_PSBL, RDNS_NONE,T_MIME_NO_TEXT,UNPARSEABLE_RELAY,URIBL_BLOCKED autolearn=no version=3.4.0-pre3-r1435395 Now, why don't I have URIBL_BLOCKED in *both*? It still seems to me that URIBL lookups are not occurring when going through the MTA, regardless of whether or not I'm blocked. Is Amavis screwing with things here, since SA is called via Amavis? --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 2:51 PM -0700 Quanah Gibson-Mount
qua...@zimbra.com wrote:
Now, why don't I have URIBL_BLOCKED in *both*? It still seems to me
that URIBL lookups are not occurring when going through the MTA,
regardless of whether or not I'm blocked. Is Amavis screwing with things
here, since SA is called via Amavis?
Yes, I see... Amavis turns off RBLs:
$spamassasin_obj = Mail::SpamAssassin-new(
{ dont_copy_prefs = 1, local_tests_only = 1 } )
That explains a lot. ;)
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 2:56 PM -0700 Quanah Gibson-Mount
qua...@zimbra.com wrote:
Now, why don't I have URIBL_BLOCKED in *both*? It still seems to me
that URIBL lookups are not occurring when going through the MTA,
regardless of whether or not I'm blocked. Is Amavis screwing with things
here, since SA is called via Amavis?
Yes, I see... Amavis turns off RBLs:
$spamassasin_obj = Mail::SpamAssassin-new(
{ dont_copy_prefs = 1, local_tests_only = 1 } )
That explains a lot. ;)
Or not... I have $sa_local_tests_only set to 0 in my amavisd.conf, so it
should be doing the URIBL tests.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Thursday, October 24, 2013 12:05 AM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-10-23 23:51: Is Amavis screwing with things here, since SA is called via Amavis? if its is, try testing spampd so its showed its not that problem, running amavis and spampd nearly is equal to postfix setup, not much time to see if amavis is at fault for this note spampd is not spamd/spamc I turned on debugging for SA at the amavis level, and I can see that periodically RBL lookups do go through, but the majority of time, it looks like VMW's dns servers are timing out our MX's. So apparently I need to go talk to VMW for a bit (and deploy a local caching name server). --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Spam constantly being autolearned as ham
We have an issue where a lot of spam is being autolearned as HAM by SA. Do people generally turn off autolearn? In looking at these cases, I'm not seeing where it is particularly helpful, but it is particularly harmful. Example: X-Spam-Status: No, score=0.348 tagged_above=-10 required=3 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_IMAGE_RATIO_02=0.437, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.8, T_HEADER_FROM_DIFFERENT_DOMAINS=0.01] autolearn=ham Authentication-Results: edge02-zcs.vmware.com (amavisd-new); dkim=pass (1024-bit key) header.d=superwebmais.com; domainkeys=fail (1024-bit key) reason=fail (message has been altered) header.from=pa...@superwebmais.com header.d=superwebmais.com Received: from edge02-zcs.vmware.com ([127.0.0.1]) by localhost (edge02-zcs.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UWg6H9T4tKVE; Tue, 22 Oct 2013 11:27:06 -0700 (PDT) Received: from c115-smtp.pumpery.com (c115-smtp.pumpery.com [5.135.12.243]) by edge02-zcs.vmware.com (Postfix) with ESMTP id 76999784 for ; Tue, 22 Oct 2013 11:27:05 -0700 (PDT) Subject: =?UTF-8?B?TmV0c2hvZXM6IFPDsyBIb2plIGF0w6kgNjAlIE9GRiBuYXMgbWVsaG9yZXMgbWFyY2FzIGUgQWRpZGFzIFNwcmluZ2JsYWRlIGVtIGF0ZSAxMnggc2VtIGp1cm9zLCBnYXJhbnRhIG8gc2V1IGFxdWk=?= Message-ID: 6a75a630dd51191df1f22605902aa...@pumpery.com Date: Tue, 22 Oct 2013 20:07:11 +0200 From: Especial Esportes pa...@superwebmais.com Reply-To: pa...@superwebmais.com --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 9:09 PM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-10-22 21:01: We have an issue where a lot of spam is being autolearned as HAM by SA. Do people generally turn off autolearn? In looking at these cases, I'm not seeing where it is particularly helpful, but it is particularly harmful. maillist is pr defination one thing all members wants, if thats not the case members would report spam to the owner of the maillist to resolve it, mostly its just disconnect to subscribed spamming user I'm not sure why you are talking about a mailing list? --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 9:16 PM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-10-22 21:11: I'm not sure why you are talking about a mailing list? vmware sends dkim signed spams ? was it a bad example ? I suggest re-reading the headers. The VMWare side was *validating* the DKIM headers on the mail because the VMWare host is what is receiving the email for delivery. The *spammer* DKIM signed their email. header.d=superwebmais.com; --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 9:28 PM +0200 Axb axb.li...@gmail.com wrote: On 10/22/2013 09:01 PM, Quanah Gibson-Mount wrote: We have an issue where a lot of spam is being autolearned as HAM by SA. Do people generally turn off autolearn? I only use autolearn - no drawbacks. assuming you are legitimately receiving this through vmware relays, add vmware's IPs to your trusted networks. That will help query BLs of IPs before the vmware hosts. I don't get the concern about VMW. The vmw hosts are *my* MTAs and in mynetworks. mail.zimbra.com - load balanced name for edge01-zcs.vmware.com, edge02-zcs.vmware.com The SPAM did not originate with my servers... It originated elsewhere. This is rather clear: Received: from c115-smtp.pumpery.com (c115-smtp.pumpery.com [5.135.12.243]) by edge02-zcs.vmware.com (Postfix) with ESMTP id 76999784 for ; Tue, 22 Oct 2013 11:27:05 -0700 (PDT) pumpery.com is the originator of this spam. I've blacklisted the from in the meantime. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 12:24 PM -0700 John Hardin jhar...@impsec.org wrote: On Tue, 22 Oct 2013, Quanah Gibson-Mount wrote: We have an issue where a lot of spam is being autolearned as HAM by SA. Do people generally turn off autolearn? In looking at these cases, I'm not seeing where it is particularly helpful, but it is particularly harmful. Example: X-Spam-Status: No, score=0.348 tagged_above=-10 required=3 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_IMAGE_RATIO_02=0.437, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.8, T_HEADER_FROM_DIFFERENT_DOMAINS=0.01] autolearn=ham What are your thresholds set to? You might want to lower your ham learning threshold and zero the RP_MATCHES_RCVD score. Thresholds are definitely enabled: v310.pre:loadplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold And it looks like we use the defaults: 10_default_prefs.cf:ifplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold 10_default_prefs.cf:bayes_auto_learn_threshold_nonspam 0.1 10_default_prefs.cf:bayes_auto_learn_threshold_spam 12.0 However, as I read the docs, the score is supposed to be lower for it to be autolearned. Last I checked, 0.348 0.1, so why was this autolearned as HAM if the cutoff is 0.1? --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 11:09 PM +0200 Axb axb.li...@gmail.com wrote: You've missed the point. mynetworks is not SA - it's Postfix and SA knows nothing about this config option. as you have SA configured, RBL lookups are done against the vmware IPs and I doubt those will be blacklisted, anywhere. If you add 208.91.0.0/22 to your SA trusted_networks (in local.cf) My SA already has trusted_networks configured as well, but you are right, this range is missing, thanks. We push the mta network bits out to all portions of the mta (postfix, amavis, SA, dspam). It looks like VMW made some IP address changes w/o notifying me. Sigh. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 11:30 PM +0200 Karsten Bräckelmann guent...@rudersport.de wrote: In other words: Non-Bayes ruleset scores may differ from the scores listed above. The score for BAYES_50 definitely needs to be subtracted. Which results in a negative score... The usefulness of RP_MATCHES_RCVD is currently under discussion. I suggest to zero out that rule, or assign it a negative zero. Ok, thanks. We'd already reduced its value recently after finding it mostly useless: score RP_MATCHES_RCVD -0.8 -0.8 -0.8 -0.8 so I'll update that to -0 --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 2:28 PM -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: --On Tuesday, October 22, 2013 11:09 PM +0200 Axb axb.li...@gmail.com wrote: You've missed the point. mynetworks is not SA - it's Postfix and SA knows nothing about this config option. as you have SA configured, RBL lookups are done against the vmware IPs and I doubt those will be blacklisted, anywhere. If you add 208.91.0.0/22 to your SA trusted_networks (in local.cf) My SA already has trusted_networks configured as well, but you are right, this range is missing, thanks. We push the mta network bits out to all portions of the mta (postfix, amavis, SA, dspam). It looks like VMW made some IP address changes w/o notifying me. Sigh. Hm, actually, never mind. My trusted_networks has 10.0.0.0/8 which covers the IP address range these resolve to in their local DNS 10.113.208.x I.e., if SA is acting off the hostname-IP mapping it gets from doing a DNS lookup or from /etc/hosts, then trusted_networks already covers the edge servers, so this shouldn't be an issue. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 12:46 AM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-10-23 00:21: Hm, actually, never mind. My trusted_networks has 10.0.0.0/8 which covers the IP address range these resolve to in their local DNS 10.113.208.x I.e., if SA is acting off the hostname-IP mapping it gets from doing a DNS lookup or from /etc/hosts, then trusted_networks already covers the edge servers, so this shouldn't be an issue. trusted_networks have nothing to do with hostnames, see here for example localhost.junc.org :) you trust 127.0.0.1 right ? Yes. ;) trusted_networks 127.0.0.0/8 10.0.0.0/8 [::1]/128 [fe80::%eth0]/64 204.14.232.64/28 204.14.234.64/28 202.129.242.65/32 96.43.144.64/32 96.43.144.65/32 96.43.148.64/32 96.43.148.65/32 182.50.78.64/28 208.91.2.22/31 --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 1:35 AM +0200 Karsten Bräckelmann guent...@rudersport.de wrote: ^^^^ 204.14.232.64/28 204.14.234.64/28 202.129.242.65/32 96.43.144.64/32 96.43.144.65/32 96.43.148.64/32 96.43.148.65/32 182.50.78.64/28 208.91.2.22/31 Excuse me for being blunt, but it appears you didn't lint check in quite a while. That is absolutely borked. $ spamassassin --lint --cf=trusted_networks 127.0.0.0/8 warn: netset: cannot include 127.0.0.0/8 as it has already been included M::SA::Conf docs, section Network Test Options, option trusted_networks states: Note: 127/8 and ::1 are always included in trusted_networks, regardless of your config. $ spamassassin --lint --cf=trusted_networks [::1]/128 warn: netset: illegal network address given: '[::1]/128' Included by default as well. And even bad syntax. However, it also does not cause harm to include the local addresses. Whether or not the syntax is bad sounds like an argument you can take to the postfix authors. Clearly their tool to generate it feels it is valid. The values themselves are generated by postfix, via postconf -d mynetworks And that last address range [fe80::%eth0]/64 on the first line is just weird -- what's supposed to substitute that ethernet interface placeholder? Generally it just gets dropped: Oct 22 12:09:24 edge02-zcs amavis[27883]: SA info: netset: ignoring interface scope '%eth0' in IP address [fe80::%eth0]/64 However, it is a leftover from a bug in postfix a while back, I've fixed that. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Tuesday, October 22, 2013 4:48 PM -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: $ spamassassin --lint --cf=trusted_networks [::1]/128 warn: netset: illegal network address given: '[::1]/128' Actually, it appears you are using an out of date spamassassin. ;) [zimbra@edge02-zcs ~]$ /opt/zimbra/zimbramon/bin/spamassassin --lint --cf=trusted_networks [::1]/128 Oct 22 16:58:40.587 [12363] warn: netset: cannot include 0:0:0:0:0:0:0:1/128 as it has already been included --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: Spam constantly being autolearned as ham
--On Wednesday, October 23, 2013 2:19 AM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-10-23 01:48: However, it is a leftover from a bug in postfix a while back, I've fixed that. bah, its not in the output of ifconfig, is it ?, if it is dont blame postfix :) No, it was literally a bug in the early postfix 2.10 development releases. I reported it back to Wietse a few years ago, but never fixed my config. ;) --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On Monday, August 12, 2013 2:02 PM -0700 John Hardin jhar...@impsec.org wrote: On Mon, 12 Aug 2013, Bowie Bailey wrote: On 8/12/2013 2:48 PM, John Hardin wrote: On Mon, 12 Aug 2013, Quanah Gibson-Mount wrote: --On Friday, August 09, 2013 12:42 AM +0200 Benny Pedersen wrote: body __BODY_FACEBOOK /Facebook/ meta __FORGED_SENDER (!SPF_PASS !DKIM_VALID_AU) meta FORGED_FACEBOOK_BODY (__BODY_FACEBOOK __FORGED_SENDER) maybe it could be more specific, just not tested it, but why accept forged ? Thanks, that is helpful. So I assume then I would do something like: score FORGED_FACEBOOK_BODY 3.0 to give it a high SPAM score. ...so you want to punish any email that discusses Facebook and does not pass SPF *AND* DKIM? Regardless of where the message is (or claims to be) from? Actually, __FORGED_SENDER only fires if the message fails *both* SPF and DKIM. (not A) and (not B) == not (A or B) D'oh! But this is still a check for message *discussing* Facebook and not messages specifically *from* Facebook. Yeah, I'm not complaining about people discussing facebook, but pretending to be facebook. Example: Return-Path: no-re...@facebook.com Received: from edge02-zcs.vmware.com (LHLO edge02-zcs.vmware.com) (10.113.208.52) by mbs01-zcs.vmware.com with LMTP; Thu, 15 Aug 2013 11:11:37 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by edge02-zcs.vmware.com (Postfix) with ESMTP id 904D1992; Thu, 15 Aug 2013 11:11:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at edge02-zcs.vmware.com X-Spam-Flag: NO X-Spam-Score: 2.814 X-Spam-Level: ** X-Spam-Status: No, score=2.814 tagged_above=-10 required=3 tests=[BAYES_80=2, DKIM_ADSP_ALL=0.8, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, KHOP_BIG_TO_CC=0.001, SPF_FAIL=0.001, T_HEADER_FROM_DIFFERENT_DOMAINS=0.01] autolearn=no Received: from edge02-zcs.vmware.com ([127.0.0.1]) by localhost (edge02-zcs.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ezz1yu95KGdl; Thu, 15 Aug 2013 11:11:36 -0700 (PDT) Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use 'no-re...@facebook.com' in 'mfrom' identity (mechanism '-all' matched)) receiver=edge02-zcs.vmware.com; identity=mailfrom; envelope-from=no-re...@facebook.com; helo=mail.oandpa.com; client-ip=68.169.160.233 Received-SPF: fail (facebook.com ... _spf.facebook.com: Sender is not authorized by default to use
RP_MATCHES_RCVD letting in SPAM
Some of our users are getting a ton of SPAM from .br domains. If it weren't for RP_MATCHES_RCVD they would actually end up in their junk folder rather than their Inbox. Is there a general suggested adjustment I can make catch these without tweaking RP_MATCHES_RCVD? Return-Path: s...@uptop.com.br Received: from edge01-zcs.vmware.com (LHLO edge01-zcs.vmware.com) (10.113.208.51) by mbs03-zcs.vmware.com with LMTP; Thu, 15 Aug 2013 11:27:16 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by edge01-zcs.vmware.com (Postfix) with ESMTP id A8C1A1931; Thu, 15 Aug 2013 11:27:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at edge01-zcs.vmware.com X-Spam-Flag: NO X-Spam-Score: 2.069 X-Spam-Level: ** X-Spam-Status: No, score=2.069 tagged_above=-10 required=3 tests=[BAYES_99=3.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, LOTS_OF_MONEY=0.001, RP_MATCHES_RCVD=-1.344, T_KHOP_FOREIGN_CLICK=0.01] autolearn=no Authentication-Results: edge01-zcs.vmware.com (amavisd-new); dkim=pass (1024-bit key) header.d=uptop.com.br Received: from edge01-zcs.vmware.com ([127.0.0.1]) by localhost (edge01-zcs.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vjdqouuXTjs0; Thu, 15 Aug 2013 11:27:15 -0700 (PDT) Received: from vmta31.uptop.com.br (vmta31.uptop.com.br [5.135.117.31]) by edge01-zcs.vmware.com (Postfix) with ESMTP id 5502699B for xx...@zimbra.com; Thu, 15 Aug 2013 11:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=upkey; d=uptop.com.br; h=To:Subject:Message-ID:Date:From:Reply-To:MIME-Version:List-Unsubscribe:Con tent-Type:Content-Transfer-Encoding; i=a...@uptop.com.br; bh=T9iP2DjK/6AQ4Vs6z6J5Ns129Jg=; b=FmrfkS17Bdb5zaJItp0+1hdmmlIoC8TXdgt/Z1/8/dPdT5K5yBka+jdLfLWKiJhR18koFcHgBl f2 5p9CbRL25dr012hmqmgH5O/auyGb2HGHNxmAv5GgthtRuCTynO2oyUJ1Ykz/fQ6wnvsReynaz8oi pj4Oy7qviqGVdBzZZ4c= To: x...@zimbra.com Subject: =?UTF-8?B?QW5pdmVyc8OhcmlvIExhIEN1aXNpbmU6IDEwJSsxMCUgZGUgRGVzY29udG8gcGFyYSBWb2PDqiA=?= Message-ID: 32c1d84426a44ac5e446b2a57d539...@www.uptop.com.br Date: Thu, 15 Aug 2013 15:08:05 -0300 From: =?UTF-8?B?U2hvcHRpbWUuY29tLmJyIC0gTcOtZGlhTWFpbA==?= m...@uptop.com.br Reply-To: m...@uptop.com.br MIME-Version: 1.0 X-Mailer-LID: 3 List-Unsubscribe: http://www.uptop.com.br/unsubscribe.php?M=1938765C=b8da7e6dcf057fc02a0cb072c0312e6fL=3N=379 X-Mailer-RecptId: 1938765 X-Mailer-SID: 379 X-Mailer-Sent-By: 1 Content-Type: multipart/alternative; charset=UTF-8; boundary=b1_bb546d207080f5562bf4cdc2c79bfd11 Content-Transfer-Encoding: 8bit -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On Thursday, August 15, 2013 3:06 PM -0400 Bowie Bailey bowie_bai...@buc.com wrote: On 8/15/2013 2:53 PM, Quanah Gibson-Mount wrote: Yeah, I'm not complaining about people discussing facebook, but pretending to be facebook. Example: Return-Path: no-re...@facebook.com Received: from edge02-zcs.vmware.com (LHLO edge02-zcs.vmware.com) (10.113.208.52) by mbs01-zcs.vmware.com with LMTP; Thu, 15 Aug 2013 11:11:37 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by edge02-zcs.vmware.com (Postfix) with ESMTP id 904D1992; Thu, 15 Aug 2013 11:11:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at edge02-zcs.vmware.com X-Spam-Flag: NO X-Spam-Score: 2.814 X-Spam-Level: ** X-Spam-Status: No, score=2.814 tagged_above=-10 required=3 tests=[BAYES_80=2, DKIM_ADSP_ALL=0.8, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, KHOP_BIG_TO_CC=0.001, SPF_FAIL=0.001, T_HEADER_FROM_DIFFERENT_DOMAINS=0.01] autolearn=no Received: from edge02-zcs.vmware.com ([127.0.0.1]) by localhost (edge02-zcs.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ezz1yu95KGdl; Thu, 15 Aug 2013 11:11:36 -0700 (PDT) snip Message-ID: 520d16e7.407...@facebook.com Date: Thu, 15 Aug 2013 13:11:34 -0500 From: Facebook notification+zrdohvri=v...@facebookmail.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101103 Thunderbird/3.1.6 MIME-Version: 1.0 So what I need is something like: header __FROM_FACEBOOK Return-Path:addr =~ /no-reply\@facebook.com/ meta __FORGED_SENDER (!SPF_PASS !DKIM_VALID_AU) meta FORGED_FACEBOOK_FROM (__FROM_FACEBOOK __FORGED_SENDER) score FORGED_FACEBOOK 1.5 Does that look correct? Looks good to me. The only thing I see is that you need to escape the period in the regex. header __FROM_FACEBOOK Return-Path:addr =~ /no-reply\@facebook\.com/ Otherwise, the period means any character, which would probably not be an issue here, but is not what you were intending. Yeah, I noticed that after I sent it, thanks. :) --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: RP_MATCHES_RCVD letting in SPAM
--On Thursday, August 15, 2013 9:16 PM +0200 Benny Pedersen wrote: Quanah Gibson-Mount skrev den 2013-08-15 21:05: Some of our users are getting a ton of SPAM from .br domains. If it weren't for RP_MATCHES_RCVD they would actually end up in their junk folder rather than their Inbox. Is there a general suggested adjustment I can make catch these without tweaking RP_MATCHES_RCVD? meta LOTS_OF_MONEY (3) (3) (3) (3) meta RP_MATCHES_RCVD (1) (1) (1) (1) Perfect, thanks! --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: RP_MATCHES_RCVD letting in SPAM
--On Thursday, August 15, 2013 12:21 PM -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: --On Thursday, August 15, 2013 9:16 PM +0200 Benny Pedersen wrote: Quanah Gibson-Mount skrev den 2013-08-15 21:05: Some of our users are getting a ton of SPAM from .br domains. If it weren't for RP_MATCHES_RCVD they would actually end up in their junk folder rather than their Inbox. Is there a general suggested adjustment I can make catch these without tweaking RP_MATCHES_RCVD? meta LOTS_OF_MONEY (3) (3) (3) (3) meta RP_MATCHES_RCVD (1) (1) (1) (1) Perfect, thanks! Hm, that won't catch our other BR spam though. :( Return-Path: reto...@registraclique.com.br Received: from edge01-zcs.vmware.com (LHLO edge01-zcs.vmware.com) (10.113.208.51) by mbs03-zcs.vmware.com with LMTP; Thu, 15 Aug 2013 11:15:55 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by edge01-zcs.vmware.com (Postfix) with ESMTP id CB83A1968; Thu, 15 Aug 2013 11:15:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at edge01-zcs.vmware.com X-Spam-Flag: NO X-Spam-Score: 2.833 X-Spam-Level: ** X-Spam-Status: No, score=2.833 tagged_above=-10 required=3 tests=[BAYES_99=3.5, DKIM_SIGNED=0.1, HTML_IMAGE_RATIO_04=0.556, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-1.344, T_DKIM_INVALID=0.01, T_KHOP_FOREIGN_CLICK=0.01] autolearn=no Authentication-Results: edge01-zcs.vmware.com (amavisd-new); dkim=neutral reason=invalid (public key: not available) header.d=registraclique.com.br Received: from edge01-zcs.vmware.com ([127.0.0.1]) by localhost (edge01-zcs.vmware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qup1pMAcaDgg; Thu, 15 Aug 2013 11:15:53 -0700 (PDT) Received: from registraclique.com.br (s175.registraclique.com.br [141.105.64.175]) by edge01-zcs.vmware.com (Postfix) with ESMTPS id 90F8A1940 for xx...@zimbra.com; Thu, 15 Aug 2013 11:15:52 -0700 (PDT) Received: by registraclique.com.br (Postfix, from userid 0) id 2BAEB8860B8; Thu, 15 Aug 2013 10:22:21 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=registraclique.com.br; s=default; t=1376590475; bh=nUoQ44WhTVHL4zF0mcmuHnMTLjLNO1sgscswqFRg/0g=; h=To:Subject:Date:From:Reply-To:List-Unsubscribe; b=ovlYK4eRDyhcbVMwLbd+TqVjdXO2pwQyko4Kc0FKjdan2k8tz9uO6y2633kIBG+fb NJLigYccPUTrD/2B6MYTgWzXulw8pQtVbXSKnuzXAq0pZmwx5a+jXiVJOWH8gsW1e7 FW+Qaxu0aIrmfOkPLOzGHALhLkg8JIxWLiAbe/lE= To: xx...@zimbra.com Subject: Fale Ilimitado Com Todo O Brasil Por R$19,90! Message-ID: 350297cb0672e79fdb9aa53472cca...@www.registraclique.com.br Date: Thu, 15 Aug 2013 09:16:29 -0400 From: =?UTF-8?B?Q2xhcm8gRmFsZSDDoCBWb250YWRl?= cont...@registraclique.com.br Reply-To: cont...@registraclique.com.br MIME-Version: 1.0 X-Mailer-LID: 11 List-Unsubscribe: http://www.registraclique.com.br/iem/unsubscribe.php?M=1531174C=77d064e695a19edb4155caf4c244402aL=11N=72 X-Mailer-RecptId: 1531174 X-Mailer-SID: 72 X-Mailer-Sent-By: 1 Content-Type: multipart/alternative; charset=UTF-8; boundary=b1_bb3d14c03992adb6a28e84dfa3fb4b7d Content-Transfer-Encoding: 8bit --b1_bb3d14c03992adb6a28e84dfa3fb4b7d Content-Type: text/plain; format=flowed; charset=UTF-8 Content-Transfer-Encoding: 8bit -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On Thursday, August 15, 2013 12:36 PM -0700 John Hardin wrote: On Thu, 15 Aug 2013, Quanah Gibson-Mount wrote: header __FROM_FACEBOOK Return-Path:addr =~ /no-reply\@facebook\.com/ Any reason you're limiting it to just the no-reply address? You might also want to broaden the domain a bit. How about: header __FROM_FACEBOOK Return-Path:addr =~ /\@facebook(?:mail)?\.com$/ well, so far, all 200 or so of these I've seen all use the same Return-Path. The From: varies, but Return-Path doesn't. --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: RP_MATCHES_RCVD letting in SPAM
--On Thursday, August 15, 2013 10:07 PM +0200 Benny Pedersen wrote: Quanah Gibson-Mount skrev den 2013-08-15 21:25: Hm, that won't catch our other BR spam though. :( List-Unsubscribe: http://www.registraclique.com.br/iem/unsubscribe.php?M=1531174C=77d064 e695a19edb4155caf4c244402aL=11N=72 unsubscribe ? if recipient was not opt-in then block sender domain with mta rule, dont accept opt-out ! Thanks Benny, I will just blacklist them. --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On Friday, August 09, 2013 12:42 AM +0200 Benny Pedersen wrote: Quanah Gibson-Mount skrev den 2013-08-08 23:22: I would love to see your rules here so I can see how you did it. I don't see if/and in the SA docs on rules. body __BODY_FACEBOOK /Facebook/ meta __FORGED_SENDER (!SPF_PASS !DKIM_VALID_AU) meta FORGED_FACEBOOK_BODY (__BODY_FACEBOOK __FORGED_SENDER) maybe it could be more specific, just not tested it, but why accept forged ? Thanks, that is helpful. So I assume then I would do something like: score FORGED_FACEBOOK_BODY 3.0 to give it a high SPAM score. --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
SPF failure very low score
For SA 3.4.0, it says in 50_scores.cf: # SPF # Note that the benefit for a valid SPF record is deliberately minimal; it's # likely that more spammers would quickly move to setting valid SPF records # otherwise. The penalties for an *incorrect* record, however, are large. ;) However, .001 does not seem LARGE to me at all. I would expect at least a 1. Right now there is tons of facebook spam out there that clearly fails SPF, such as the following: X-Spam-Status: No, score=2.407 tagged_above=-10 required=3 tests=[BAYES_50=0.8, DKIM_ADSP_ALL=0.8, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, KHOP_BIG_TO_CC=0.001, RDNS_NONE=0.793, SPF_FAIL=0.001, T_HEADER_FROM_DIFFERENT_DOMAINS=0.01] autolearn=no How is .001 in any way considered a large penalty? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On August 8, 2013 1:49:18 PM -0700 John Hardin jhar...@impsec.org wrote: How is .001 in any way considered a large penalty? SPF is _by itself_ not useful as a spam sign. If you're seeing a lot of facebook spam that fails SPF because it's being forged, then a rule that checks SPF_FAIL *IF* the mail claims to be from Facebook, and adds a point or two, would be more reasonable. Ok, that sounds reasonable, but that still doesn't align with the comment in the 50_scores.cf file. ;) Can you provide an example? I've done some basic custom rules, but the above is a little more complex. Thanks, Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On August 8, 2013 5:14:12 PM -0400 David F. Skoll d...@roaringpenguin.com wrote: On Thu, 8 Aug 2013 13:49:18 -0700 (PDT) John Hardin jhar...@impsec.org wrote: SPF is _by itself_ not useful as a spam sign. Indeed. In my experience, most SPF softfail results and a fairly large fraction of SPF fail results are from misconfigured domains whose administrators don't bother making correct SPF records. Additionally, SPF pass is (in my experience) a slight indicator of spam because spammers are a bit more diligent about trying to get their messages to pass SPF than many legitimate senders. :( +1 to John's comments about domain-specific SPF scores. For certain domains, an SPF fail is a strong indicator of spam or phishing. These are the domains I score strongly for SPF fail: adp.com, aexp.com, apple.com, bankofamerica.com, bbb.org, bmo.com, chase.com, discover.com, dnb.com, ebay.com, emailinfo.chase.com, id.apple.com, inbound.efax.com, irs.gov, newegg.com, paypal.com, verizonwireless.com, welcome.aexp.com, wellsfargo.com as well as my own domain, roaringpenguin.com. I would love to see your rules here so I can see how you did it. I don't see if/and in the SA docs on rules. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On August 8, 2013 5:38:52 PM -0400 dar...@chaosreigns.com wrote: The explanation for the quote is, quite simply, that it is out of date, and you should fix it. I don't have commit access to SA's SVN. ;) I suppose I can file a bug. ;) --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On August 8, 2013 5:33:26 PM -0400 David F. Skoll d...@roaringpenguin.com wrote: On Thu, 08 Aug 2013 14:22:53 -0700 Quanah Gibson-Mount qua...@zimbra.com wrote: I would love to see your rules here so I can see how you did it. I don't see if/and in the SA docs on rules. Emm... actually, I did it outside of the SA infrastructure. I imagine you could do something like: header__MY_SENSITIVE_DOMAIN Return-Path =~ /\@(:?ebay\.com|paypal\.com|irs\.gov)/i meta MY_SPF_FAIL SPF_FAIL __MY_SENSITIVE_DOMAIN score MY_SPF_FAIL 5.0 describe MY_SPF_FAIL SPF failure on a sensitive domain Thanks, that's a useful start. :) --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: SPF failure very low score
--On August 8, 2013 11:01:43 PM +0100 RW rwmailli...@googlemail.com wrote:
Facebook dkim signs all their emails with the domain
facebookmail.com, so you may have better luck using the ADSP rules...
dkim is generally the better way to go since legitimate emails can fail
SPF due to forwarding.
Ok, so I imagine I want to do something like:
header DKIM_ADSP_DISCARD eval:check_dkim_adsp('D')
but only for facebook.com... I don't see exactly how I tie those two
together?
Thanks!
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
Getting AWL to unlearn (SA 3.4.0 2013/04/01)
Running SA 3.4.0 from April 1st, 2013. I'm seeing an issue where obvious spam is reporting as whitelisted from SA for some users. We do not have per-user whitelisting, so it seems AWL as for some unknown and bizarre reason, decided to whitelist this spam. The *same* emails for me get marked as spam correctly. Example scoring: X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=9.734 tagged_above=-10 required=3 WHITELISTED tests=[BAYES_99=3.5, FSL_HELO_BARE_IP_1=2.347, FSL_HELO_BARE_IP_2=1.738, LONG_TERM_PRICE=0.001, RCVD_NUMERIC_HELO=1.164, RDNS_DYNAMIC=0.982, TVD_RCVD_IP=0.001, TVD_RCVD_IP4=0.001] autolearn=no No matter how much I feed these emails to SA for training as spam, the user continues to have them show up whitelisted. I know I can disable AWL, but is there any way to clear specific bits out of AWL so anything valid it has picked up doesn't get lost? Thanks, Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
Re: Getting AWL to unlearn (SA 3.4.0 2013/04/01)
--On Monday, July 29, 2013 10:19 PM +0200 Benny Pedersen m...@junc.eu wrote: Quanah Gibson-Mount skrev den 2013-07-29 21:50: X-Spam-Status: No, score=9.734 tagged_above=-10 required=3 WHITELISTED No matter how much I feed these emails to SA for training as spam, the user its not whitelisted in sa, its amavisd dont blame sa for this :) Ah, ok, thanks. ;) Which is odd too, but I have some ideas then to pursue at least. Thanks! --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration
