Re: Config option to skip pyzor check on empty body emails?
On 12/09/17 12:33, RW wrote: On Tue, 12 Sep 2017 08:41:01 +0100 Sebastian Arcus wrote: The confusing part is that left to its devices, Pyzor creates a .pyzor dir in the home dir of the user it is run as. But if --homedir is specified, it dumps stuff directly there, instead of creating a .pyzor dir.In the end I got rid of the "pyzor_options --homedir" option in local.cf and it worked fine. It is a bit confusing, but it's not that the .pyzor directory is use inconsistently, it's that pyzor defines --homedir=HOMEDIR configuration directory so the default homedir is $HOME/.pyzor/ not $HOME/. If you want to use pyzor_options you could use: pyzor_options --homedir /var/spool/spamd/.pyzor Like with everything, it all makes sense after you fully understand what's going on :-) I just made the wrong assumptions about how the option would work. Like Ian says, the word "home" in the option name makes it easy to assume that everything will be arranged as subdirectories under it. No matter - I'm happy I've finally found a solution to the empty bodied emails hitting PYZOR_CHECK :-) Thanks again for all the help.
Re: Config option to skip pyzor check on empty body emails?
On 2017-09-12 12:33, RW wrote: > It is a bit confusing, but it's not that the .pyzor directory is use > inconsistently, it's that pyzor defines > > --homedir=HOMEDIR configuration directory The confusing part is the spelling of the option. The mistake is clear from the last line quoted above: it should be "configdir" and not "homedir". Admittedly pyzor will put the data there by default as well (when backed by gdbm) but that's a minor quibble by comparison. -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. Do obvious transformation on domain to reply privately _only_ on Usenet.
Re: Config option to skip pyzor check on empty body emails?
On Tue, 12 Sep 2017 08:41:01 +0100 Sebastian Arcus wrote: > The confusing part is that left to its devices, Pyzor creates > a .pyzor dir in the home dir of the user it is run as. But if > --homedir is specified, it dumps stuff directly there, instead of > creating a .pyzor dir.In the end I got rid of the "pyzor_options > --homedir" option in local.cf and it worked fine. It is a bit confusing, but it's not that the .pyzor directory is use inconsistently, it's that pyzor defines --homedir=HOMEDIR configuration directory so the default homedir is $HOME/.pyzor/ not $HOME/. If you want to use pyzor_options you could use: pyzor_options --homedir /var/spool/spamd/.pyzor
Re: Config option to skip pyzor check on empty body emails?
On 12/09/17 00:56, RW wrote: On Tue, 12 Sep 2017 00:37:40 +0100 Sebastian Arcus wrote: On 11/09/17 20:20, RW wrote: This is why pyzor has the local_whitelist command. At very least it's a good idea to pipe an empty string through "pyzor local_whitelist" (probably as the user running spamassassin). I have spotted that command in the docs - and if it worked, it would seem like a good solution. But it doesn't seem to. I have added the hash of the empty string to the local whitelist. If I try to re-add the same hash, or the hash of the problem emails - I get a message stating that it is already in the whitelist - so it would appear to be working. But when running the email message through SA, it still hits PYZOR_CHECK. I have found the location of Pyzor's local whitelist - and the permissions are correct. It appears that SA completely ignores the fact that the digest is whitelisted locally: SA can't ignore it, if a hash is whitelisted pyzor returns a dummy result. e.g.: $ echo "" | pyzor check public.pyzor.org:24441 (200, 'OK') 0 0 compared with: $ echo "" | pyzor --local-whitelist=/nonextistent check public.pyzor.org:24441 (200, 'OK') 2749671 82562 Thank you for that. I finally gotten to the bottom of my problem. It was the Pyzor homedir. Although I have set it up in /etc/mail/spamassassin/local.cf, I ended up confusing myself. If I ran as root: #pyzor local_whitelist < /email.eml it placed the whitelist in /root/.pyzor/whitelist When I ran: #su - spamd -c "pyzor local_whitelist < /email.eml" it placed it in /var/spool/spamd/.pyzor/whitelist (/var/spool/spamd is the homedir of the 'spamd' user on this system) But when I ran: #su - spamd -c "pyzor --homedir /var/spool/spamd < /email.eml" it placed it in /var/spool/spamd/whitelist The confusing part is that left to its devices, Pyzor creates a .pyzor dir in the home dir of the user it is run as. But if --homedir is specified, it dumps stuff directly there, instead of creating a .pyzor dir. In the end I got rid of the "pyzor_options --homedir" option in local.cf and it worked fine. I was just tying myself in knots there :-) Thanks again
Re: Config option to skip pyzor check on empty body emails?
On Tue, 12 Sep 2017 00:37:40 +0100 Sebastian Arcus wrote: > On 11/09/17 20:20, RW wrote: > > This is why pyzor has the local_whitelist command. At very least > > it's a good idea to pipe an empty string through > > "pyzor local_whitelist" (probably as the user running > > spamassassin). > > I have spotted that command in the docs - and if it worked, it would > seem like a good solution. But it doesn't seem to. I have added the > hash of the empty string to the local whitelist. If I try to re-add > the same hash, or the hash of the problem emails - I get a message > stating that it is already in the whitelist - so it would appear to > be working. But when running the email message through SA, it still > hits PYZOR_CHECK. I have found the location of Pyzor's local > whitelist - and the permissions are correct. It appears that SA > completely ignores the fact that the digest is whitelisted locally: SA can't ignore it, if a hash is whitelisted pyzor returns a dummy result. e.g.: $ echo "" | pyzor check public.pyzor.org:24441 (200, 'OK') 0 0 compared with: $ echo "" | pyzor --local-whitelist=/nonextistent check public.pyzor.org:24441 (200, 'OK') 2749671 82562
Re: Config option to skip pyzor check on empty body emails?
On 11/09/17 20:20, RW wrote: On Mon, 11 Sep 2017 17:39:16 +0100 Sebastian Arcus wrote: Is there any way to tell SA to skip pyzor checks on emails with an empty body (even if there are attachments). I've noticed for a while now that emails which don't contain any text in their bodies seem to automatically trigger PYZOR_CHECK (even if they have an attachment) - although they are private emails so can't possibly match the digest of spam emails. I can only guess that Pyzor matches the digest of empty emails automatically. It's because pyzor is based only on a simplified version of the body text. This includes stripping any URIs or email addresses from the text. It's not just emails with no body text there are also variants of this that reduce to common phrases such as "Sent from my iPhone" I have clients who receive important emails from their customers just with an attachment and a subject line - and they all seem to go to Junk - because they trigger the PYZOR_CHECK rule - which is causing problems. Any way to deal with this? This is why pyzor has the local_whitelist command. At very least it's a good idea to pipe an empty string through "pyzor local_whitelist" (probably as the user running spamassassin). I have spotted that command in the docs - and if it worked, it would seem like a good solution. But it doesn't seem to. I have added the hash of the empty string to the local whitelist. If I try to re-add the same hash, or the hash of the problem emails - I get a message stating that it is already in the whitelist - so it would appear to be working. But when running the email message through SA, it still hits PYZOR_CHECK. I have found the location of Pyzor's local whitelist - and the permissions are correct. It appears that SA completely ignores the fact that the digest is whitelisted locally: su - spamd -c "spamassassin -D 2>&1 < /test1.eml" | grep -i pyzor Sep 12 00:31:49.080 [23559] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC Sep 12 00:31:49.090 [23559] dbg: pyzor: network tests on, attempting Pyzor Sep 12 00:31:50.679 [23559] dbg: config: fixed relative path: /var/lib/spamassassin/3.004001/updates_spamassassin_org/25_pyzor.cf Sep 12 00:31:50.679 [23559] dbg: config: using "/var/lib/spamassassin/3.004001/updates_spamassassin_org/25_pyzor.cf" for included file Sep 12 00:31:50.680 [23559] dbg: config: read file /var/lib/spamassassin/3.004001/updates_spamassassin_org/25_pyzor.cf Sep 12 00:31:57.411 [23559] dbg: util: executable for pyzor was found at /usr/bin/pyzor Sep 12 00:31:57.412 [23559] dbg: pyzor: pyzor is available: /usr/bin/pyzor Sep 12 00:31:57.413 [23559] dbg: pyzor: opening pipe: /usr/bin/pyzor --homedir /var/spool/spamd check < /tmp/.spamassassin23559DIrl4Ktmp Sep 12 00:31:58.154 [23559] dbg: pyzor: [23560] finished: exit 1 Sep 12 00:31:58.155 [23559] dbg: pyzor: got response: public.pyzor.org:24441 (200, 'OK') 2749542 82562 Sep 12 00:31:58.156 [23559] dbg: check: tagrun - tag PYZOR is now ready, value: Whitelisted. Sep 12 00:31:58.157 [23559] dbg: pyzor: listed: COUNT=2749542/5 WHITELIST=82562 Sep 12 00:31:58.159 [23559] dbg: rules: ran eval rule PYZOR_CHECK ==> got hit (1) * 2.5 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/) 2.5 PYZOR_CHECKListed in Pyzor (http://pyzor.sf.net/)
Re: Config option to skip pyzor check on empty body emails?
On Mon, 11 Sep 2017 17:39:16 +0100 Sebastian Arcus wrote: > Is there any way to tell SA to skip pyzor checks on emails with an > empty body (even if there are attachments). I've noticed for a while > now that emails which don't contain any text in their bodies seem to > automatically trigger PYZOR_CHECK (even if they have an attachment) - > although they are private emails so can't possibly match the digest > of spam emails. I can only guess that Pyzor matches the digest of > empty emails automatically. It's because pyzor is based only on a simplified version of the body text. This includes stripping any URIs or email addresses from the text. It's not just emails with no body text there are also variants of this that reduce to common phrases such as "Sent from my iPhone" > I have clients who receive important > emails from their customers just with an attachment and a subject > line - and they all seem to go to Junk - because they trigger the > PYZOR_CHECK rule - which is causing problems. Any way to deal with > this? This is why pyzor has the local_whitelist command. At very least it's a good idea to pipe an empty string through "pyzor local_whitelist" (probably as the user running spamassassin).
Re: Config option to skip pyzor check on empty body emails?
On 09/11/2017 11:39 AM, Sebastian Arcus wrote: Is there any way to tell SA to skip pyzor checks on emails with an empty body (even if there are attachments). I've noticed for a while now that emails which don't contain any text in their bodies seem to automatically trigger PYZOR_CHECK (even if they have an attachment) - although they are private emails so can't possibly match the digest of spam emails. I can only guess that Pyzor matches the digest of empty emails automatically. I have clients who receive important emails from their customers just with an attachment and a subject line - and they all seem to go to Junk - because they trigger the PYZOR_CHECK rule - which is causing problems. Any way to deal with this? I mostly solved this with a meta rule to subtract points when this combination is hit: metaENA_EMPTY_BODY (DCC_CHECK || PYZOR_CHECK || DIGEST_MULTIPLE) && (TVD_SPACE_RATIO || GMD_PDF_EMPTY_BODY) && (SPF_PASS || DMARC_PASS || ALL_TRUSTED) && !MISSING_HEADERS describeENA_EMPTY_BODY Empty body emails with only attachments. score ENA_EMPTY_BODY -3.2 DMARC_PASS is a local rule from a local header added by OpenDMARC so that will probably need to be removed. -- David Jones
Config option to skip pyzor check on empty body emails?
Is there any way to tell SA to skip pyzor checks on emails with an empty body (even if there are attachments). I've noticed for a while now that emails which don't contain any text in their bodies seem to automatically trigger PYZOR_CHECK (even if they have an attachment) - although they are private emails so can't possibly match the digest of spam emails. I can only guess that Pyzor matches the digest of empty emails automatically. I have clients who receive important emails from their customers just with an attachment and a subject line - and they all seem to go to Junk - because they trigger the PYZOR_CHECK rule - which is causing problems. Any way to deal with this?
