Re: spamass-milter (sa daemon loads config different to shell ?)
On Tue, 27 Jul 2021, David Bürgin wrote:
Dipl-Inform. Frank Gadegast:
On 27.07.21 14:18, David Bürgin wrote:
Dipl-Inform. Frank Gadegast:
Seems to be, that spamass-milter simply strippes out any X-Spam* header
lines, not caring, if the own call to spamd sets them, hm.
Im really not getting, why spamass-milter should strip X-Spam-lines of
the header AFTER SA was running. If Im right, SA is stripping them of
anyway, before running or modifying anything ...
Anybody an idea how to get arround this ?
There is an alternative milter (which I maintain) that adds
all X-Spam-* headers received from spamd.
https://crates.io/crates/spamassassin-milter
Looks like your milter needs to fork a spamc, wich then talks to the spamd.
This will start lots of spamc processes and is not recommened.
Would then not be any different to call spamc dirctly f.e. via procmail.
You should rewrite your milter to talk directly to the spamd via socket or
port.
Yes, it communicates using spamc, just like spamass-milter.
I have been told that it has been working fine in a somewhat larger
deployment. I didn’t mean to derail the thread so will leave it at that.
having a spam filtering milter fork off a shell and then run "spamc" to
communicate with spamd does simplify the milter code (and insulates it from
changes in the spamd protocol) but adds risk of shell escape attacks (as well as
additional overhead).
There's already been security related patches needed by spamass-milter
specifically because of this issue.
Writing a milter that directly talks the spamd protocol via a socket (local or
network) is more work but safer and more efficient.
(been there, done that, got the code to prove it).
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-05491256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
Re: DKIM_* scores
On Mon, 26 Jul 2021, RW wrote: On Mon, 26 Jul 2021 18:05:35 +0100 RW wrote: "&& !DKIM_SIGNED " means the rule can only be true if there's no signature, so none of the terms with __DKIM_DEPENDABLE, DKIM_VALID, and DKIM_VALID_AU make any difference. Actually it's worse than that __DKIM_DEPENDABLE is always true if there are no signatures, so !DKIM_SIGNED && !__DKIM_DEPENDABLE is always false. Thanks for pointing that out. Those are "FP exclusions", not part of the base rule logic - generated by inspecting the rulequ results and excluding hits on other rules where the combination is hammy and not (or very weakly, like 1%) spammy. The interactions of combinations of those exclusions isn't considered. They also need to be reviewed periodically, which I'm doing now for XPRIO. __DKIM_DEPENDABLE is no longer a useful FP exclusion for XPRIO, as it hits 100% of the spam hits. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.org pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Maxim IX: Never turn your back on an enemy. --- 8 days until the 286th anniversary of John Peter Zenger's acquittal
Re: sa daemon loads config different to shell ?
Dipl-Inform. Frank Gadegast: On 27.07.21 14:18, David Bürgin wrote: Dipl-Inform. Frank Gadegast: Seems to be, that spamass-milter simply strippes out any X-Spam* header lines, not caring, if the own call to spamd sets them, hm. Im really not getting, why spamass-milter should strip X-Spam-lines of the header AFTER SA was running. If Im right, SA is stripping them of anyway, before running or modifying anything ... Anybody an idea how to get arround this ? There is an alternative milter (which I maintain) that adds all X-Spam-* headers received from spamd. https://crates.io/crates/spamassassin-milter Looks like your milter needs to fork a spamc, wich then talks to the spamd. This will start lots of spamc processes and is not recommened. Would then not be any different to call spamc dirctly f.e. via procmail. You should rewrite your milter to talk directly to the spamd via socket or port. Yes, it communicates using spamc, just like spamass-milter. I have been told that it has been working fine in a somewhat larger deployment. I didn’t mean to derail the thread so will leave it at that. Ciao, David
Re: DKIM_* scores
On Mon, 26 Jul 2021 08:08:10 -0400 Greg Troxel wrote: So -0.2 means that there are two dkim signatures, one for each, and they are both valid. On 26.07.21 18:05, RW wrote: It could do, but usually it just means that the sender and author domains are the same. > BTW, looking at metas in 72_active.cf: > > meta XPRIO __XPRIO_MINFP && !DKIM_SIGNED && > !__DKIM_DEPENDABLE && !DKIM_VALID && !DKIM_VALID_AU && > !RCVD_IN_DNSWL_NONE meta XPRIO __XPRIO_MINFP && > !DKIM_SIGNED && !__DKIM_DEPENDABLE && !DKIM_VALID && !DKIM_VALID_AU > && !RCVD_IN_DNSWL_NONE && !SPF_PASS > > !DKIM_VALID && !DKIM_VALID_AU is redundant and !DKIM_VALID_AU > should be enough I don't think so. These are negated. if there's no valid signature, there can't be valid author domain signature. If there's valid author domain signature, there's surely at least valid signature. imho we should compare author domain signature, not any (random) signature. "&& !DKIM_SIGNED " means the rule can only be true if there's no signature, so none of the terms with __DKIM_DEPENDABLE, DKIM_VALID, and DKIM_VALID_AU make any difference. It's usually not a good idea to use DKIM_SIGNED because it relies on the plugin, whereas __DKIM_EXISTS and the duplicate rule __HAS_DKIM_SIGHD don't. yes, more rules are kinda redundant here !DKIM_SIGNED && !__DKIM_DEPENDABLE && !DKIM_VALID && !DKIM_VALID_AU if message is not signed, then signature can't be valid or invalid. If any of signatures is valid, the message is signed. the !DKIM_SIGNED is useless here unless it's a performance optimization. Is it? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Silvester Stallone: Father of the RISC concept.
Re: DKIM_* scores
On 26.07.21 08:40, Kevin A. McGrail wrote: Correct. The fact that there are some scores that add up to approximately -0.2 is negligible when compared to a standard threshold of 5.0. Do you have false positives being caused by these emails? Do you have false negatives? That's more important to look at then just focusing on one set of rules. to be more precise, I have case where these caused mail to be autolearned as ham which is even worse than a FN I tried to filter out other rules that could cause it. Unfortunately no other rules hit that could avoid trainin. Matus UHLAR - fantomas writes: > I noticed that pure existence of DKIM signature can push score under zero: > > DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, > > ...so the cumulative score is -0.2. > > I'm aware that we don't have many rules with negative scores, but multiple > scores for single valid DKIM sinature should not be redundant. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I just got lost in thought. It was unfamiliar territory.
Re: sa daemon loads config different to shell ?
On 27.07.21 14:18, David Bürgin wrote: There is an alternative milter (which I maintain) that adds all X-Spam-* headers received from spamd. Matus UHLAR - fantomas: the original milter does the same. Adds headers from spamd. However, it does NOT take into account ay X-Spam-* headers received from remote server. On 27.07.21 15:29, David Bürgin wrote: The point is that spamass-milter does not add custom headers like ‘X-Spam-ASN’ received from spamd, only a few select ones. The alternative milter on the other hand adds all X-Spam- headers. This is also what’s discussed in the linked GitHub issue. sorry then, I think I mistook this with other thread... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. WinError #9: Out of error messages.
Re: sa daemon loads config different to shell ?
Matus UHLAR - fantomas: On 27.07.21 14:18, David Bürgin wrote: There is an alternative milter (which I maintain) that adds all X-Spam-* headers received from spamd. the original milter does the same. Adds headers from spamd. However, it does NOT take into account ay X-Spam-* headers received from remote server. The point is that spamass-milter does not add custom headers like ‘X-Spam-ASN’ received from spamd, only a few select ones. The alternative milter on the other hand adds all X-Spam- headers. This is also what’s discussed in the linked GitHub issue.
Re: sa daemon loads config different to shell ?
Dipl-Inform. Frank Gadegast: Seems to be, that spamass-milter simply strippes out any X-Spam* header lines, not caring, if the own call to spamd sets them, hm. Im really not getting, why spamass-milter should strip X-Spam-lines of the header AFTER SA was running. If Im right, SA is stripping them of anyway, before running or modifying anything ... Anybody an idea how to get arround this ? On 27.07.21 14:18, David Bürgin wrote: There is an alternative milter (which I maintain) that adds all X-Spam-* headers received from spamd. the original milter does the same. Adds headers from spamd. However, it does NOT take into account ay X-Spam-* headers received from remote server. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. REALITY.SYS corrupted. Press any key to reboot Universe.
Re: sa daemon loads config different to shell ?
Dipl-Inform. Frank Gadegast: Seems to be, that spamass-milter simply strippes out any X-Spam* header lines, not caring, if the own call to spamd sets them, hm. Im really not getting, why spamass-milter should strip X-Spam-lines of the header AFTER SA was running. If Im right, SA is stripping them of anyway, before running or modifying anything ... Anybody an idea how to get arround this ? There is an alternative milter (which I maintain) that adds all X-Spam-* headers received from spamd. https://crates.io/crates/spamassassin-milter
Re: sa daemon loads config different to shell ?
> Jul 26 23:04:41 topixx2 spamd[22655]: asn: no first external relay IP available, skipping ASN check the plugin seems to be loaded but as no external relay IP found there is nothing to check. Ensure that your trusted_networks and/or internal_networks is set correctly. Cheers tobi On 7/27/21 10:44 AM, Dipl-Inform. Frank Gadegast wrote: > > Hi, > > Im using SA 3.4 on CentOS 7 via milter. > And have a asn.cf in /etc/mail/spamassassin like this: > > loadplugin Mail::SpamAssassin::Plugin::ASN > asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_ > add_header all ASN _ASN_ _ASNCIDR_ > > This gets ignored, when sendmail runs mails through spamass-milter and > spamd, there is no X-Spam-ASN nor a X-Spam-Source header inserted. > > But it works, when filtering a mail manually via shell like this > > # spamassassin < > resulting in the expected headers: > X-Spam-ASN: AS5669 194.77.152.0/23 > X-Spam-Source: 194.77.152.124 > > So I started the spamd in debug-mode to track this down and it logged > this: > > Jul 26 23:04:40 topixx2 spamd[22655]: config: read file > /etc/mail/spamassassin/asn.cf > Jul 26 23:04:40 topixx2 spamd[22655]: config: fixed relative path: > /var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf > Jul 26 23:04:40 topixx2 spamd[22655]: config: using > "/var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf" > for included file > Jul 26 23:04:40 topixx2 spamd[22655]: config: read file > /var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf > Jul 26 23:04:40 topixx2 spamd[22655]: plugin: loading > Mail::SpamAssassin::Plugin::ASN from @INC > Jul 26 23:04:41 topixx2 spamd[22655]: plugin: > Mail::SpamAssassin::Plugin::ASN=HASH(0x3330530) implements > 'parsed_metadata', priority 0 > Jul 26 23:04:41 topixx2 spamd[22655]: asn: no first external relay IP > available, skipping ASN check > > Im I right, that it looks like, if the setting in my asn.cf could be > overwritten with the default ? > > Other .cf-files located in /etc/mail/spamassassin load all right and > show the estimated result, only asn.cf somehow fails. > > > Where should I look to find the differences in the config of > spamassassin and spamd ? > > > > > Mit freundlichen Gruessen, Frank Gadegast > -- > PHADE Software - PowerWeb http://www.powerweb.de > Inh. Dipl.-Inform. Frank Gadegast mailto:fr...@powerweb.de > Schinkelstrasse 17 fon: +49 33200 52920 > 14558 Nuthetal OT Rehbruecke, Germany fax: +49 33200 52921 > == > OpenPGP_signature Description: OpenPGP digital signature
Re: sa daemon loads config different to shell ?
On 27.07.21 11:18, Reindl Harald wrote: Hi Harald, Am 27.07.21 um 10:44 schrieb Dipl-Inform. Frank Gadegast: Im using SA 3.4 on CentOS 7 via milter. And have a asn.cf in /etc/mail/spamassassin like this: loadplugin Mail::SpamAssassin::Plugin::ASN asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_ add_header all ASN _ASN_ _ASNCIDR_ This gets ignored, when sendmail runs mails through spamass-milter and spamd, there is no X-Spam-ASN nor a X-Spam-Source header inserted the problem here is that spamass-milter has it's own idea of which headers are present - god only knows why your asn.cf is running fine ... Yes, I tweaked the asn.cf to create a test that will be listed in the X-Spam-Status. The asn.cf really works, even when loading via spamass-milter ... > but spamass-milter throws away the header ... also right. spamass-milter is removing the header lines. Yes, I found this here https://github.com/andybalholm/spamass-milter/issues/11 Seems to be, that spamass-milter simply strippes out any X-Spam* header lines, not caring, if the own call to spamd sets them, hm. Im really not getting, why spamass-milter should strip X-Spam-lines of the header AFTER SA was running. If Im right, SA is stripping them of anyway, before running or modifying anything ... Anybody an idea how to get arround this ? -- Mit freundlichen Gruessen, Frank Gadegast -- MOTD: "have you enabled SSL on a website or mailbox today ?" -- PHADE Software - PowerWeb http://www.powerweb.de Inh. Dipl.-Inform. Frank Gadegast mailto:fr...@powerweb.de Schinkelstrasse 17fon: +49 33200 52920 14558 Nuthetal OT Rehbruecke, Germany fax: +49 33200 52921 == smime.p7s Description: S/MIME Cryptographic Signature
sa daemon loads config different to shell ?
Hi, Im using SA 3.4 on CentOS 7 via milter. And have a asn.cf in /etc/mail/spamassassin like this: loadplugin Mail::SpamAssassin::Plugin::ASN asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_ add_header all ASN _ASN_ _ASNCIDR_ This gets ignored, when sendmail runs mails through spamass-milter and spamd, there is no X-Spam-ASN nor a X-Spam-Source header inserted. But it works, when filtering a mail manually via shell like this # spamassassin < resulting in the expected headers: X-Spam-ASN: AS5669 194.77.152.0/23 X-Spam-Source: 194.77.152.124 So I started the spamd in debug-mode to track this down and it logged this: Jul 26 23:04:40 topixx2 spamd[22655]: config: read file /etc/mail/spamassassin/asn.cf Jul 26 23:04:40 topixx2 spamd[22655]: config: fixed relative path: /var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf Jul 26 23:04:40 topixx2 spamd[22655]: config: using "/var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf" for included file Jul 26 23:04:40 topixx2 spamd[22655]: config: read file /var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf Jul 26 23:04:40 topixx2 spamd[22655]: plugin: loading Mail::SpamAssassin::Plugin::ASN from @INC Jul 26 23:04:41 topixx2 spamd[22655]: plugin: Mail::SpamAssassin::Plugin::ASN=HASH(0x3330530) implements 'parsed_metadata', priority 0 Jul 26 23:04:41 topixx2 spamd[22655]: asn: no first external relay IP available, skipping ASN check Im I right, that it looks like, if the setting in my asn.cf could be overwritten with the default ? Other .cf-files located in /etc/mail/spamassassin load all right and show the estimated result, only asn.cf somehow fails. Where should I look to find the differences in the config of spamassassin and spamd ? Mit freundlichen Gruessen, Frank Gadegast -- PHADE Software - PowerWeb http://www.powerweb.de Inh. Dipl.-Inform. Frank Gadegast mailto:fr...@powerweb.de Schinkelstrasse 17fon: +49 33200 52920 14558 Nuthetal OT Rehbruecke, Germany fax: +49 33200 52921 == smime.p7s Description: S/MIME Cryptographic Signature
