[Bug 8000] AMAZON_IMG_NOT_RCVD_AMZN rule producing false positives

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8000

John Hardin  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |WORKSFORME
 CC||jhar...@impsec.org

--- Comment #1 from John Hardin  ---
I cannot reproduce this FP in my SA test environment with the attached example.

May 30 20:07:11.714 [21295] dbg: rules-all: running header rule
__HDR_RCVD_AMAZON
May 30 20:07:11.714 [21295] dbg: rules: ran header rule __HDR_RCVD_AMAZON
==> got hit: " rdns=a13-123.smtp-out.amazonses.com "
...
May 30 20:07:12.731 [21295] dbg: rules-all: ran meta rule
__AMAZON_IMG_NOT_RCVD_AMZN, no hit
...
May 30 20:07:12.742 [21295] dbg: rules-all: ran meta rule
AMAZON_IMG_NOT_RCVD_AMZN, no hit


It's possible that the local-MTA Received header which prevents it from hitting
is not being added to the message by your MTA before it's being passed to
SpamAssassin for scanning. How is SpamAssassin glued onto your MTA?

Recommendations:
(1) whitelist Amazon (see below), these messages pass DKIM
(2) look into rule LOCAL_RND_SUBJ, that contributed more to the FP than
AMAZON_IMG_NOT_RCVD_AMZN did
(3) follow up the MTA glue question on the SpamAssasssin Users mailing list,
you'll get better results for rules questions there.

Suggested Amazon authenticated whitelisting:

whitelist_auth  *@amazon.com
blacklist_from  *@amazon.com
whitelist_auth  *@*.amazon.com
blacklist_from  *@*.amazon.com

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7986] spamd test fails when in directory with absolute pathname length greater than 50

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7986

--- Comment #15 from Henrik Krohns  ---
(In reply to Sidney Markowitz from comment #13)
> Committed revision 1901434.

You are testing for wrong -d $workdir

+  $socketdir = tempdir(CLEANUP => 1);
+  die "FATAL: failed to create workdir: $!" unless -d $workdir;

Also I kind of don't like the unnecessary creation of $socketdir when 90% of
the tests don't actually need it. Function similar to mk_safe_tmpdir would be
preferable to me.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7986] spamd test fails when in directory with absolute pathname length greater than 50

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7986

Henrik Krohns  changed:

   What|Removed |Added

 Resolution|FIXED   |---
 Status|RESOLVED|REOPENED

--- Comment #14 from Henrik Krohns  ---
Fine with me to use tempdir for spamd.sock only.

As you mentioned, it would use a disgusting pathname on Mac by default, but it
should be short enough? So no need to bother testing for writable /tmp /var/tmp
/dev/shm dirs?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7986] spamd test fails when in directory with absolute pathname length greater than 50

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7986

Sidney Markowitz  changed:

   What|Removed |Added

 Status|REOPENED|RESOLVED
 Resolution|--- |FIXED

--- Comment #13 from Sidney Markowitz  ---
trunk % svn ci -m "bug 7986 - Fix by using File::Temp::tempdir() for socketpath
in tests"
Sendingt/SATest.pm
Sendingt/spamc_cf.t
Sendingt/spamd_client.t
Sendingt/spamd_plugin.t
Sendingt/spamd_unix.t
Sendingt/spamd_unix_and_tcp.t
Transmitting file data ..done
Committing transaction...
Committed revision 1901434.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7986] spamd test fails when in directory with absolute pathname length greater than 50

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7986

--- Comment #12 from Sidney Markowitz  ---
Is there any reason not to use File::Temp::tempdir(CLEANUP => 1) to create a
temporary directory to use for the socket path during the tests? I can see that
we want $workdir created in SATest.pm to be in the test's log directory so we
can conveniently look at test errors. But there is no reason for the tests to
use that for $sockpath, which is the only time the path length is a problem.

In other words, for the various tests that were patched in revision 1900586
change

my $sockpath = $workdir."/spamd.sock";

into something like

my $tempdir = File::Temp::tempdir(CLEANUP => 1);
my $sockpath = $tempdir."/spamd.sock";

Or perhaps define mk_safe_tmpdir() in SATest.pm again, but a smaller one that
just returns File::Temp::tempdir(CLEANUP => 1) and call that in the tests.

$workdir could keep its existing definition for all other uses.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7981] Remaining tasks for 4.0.0 release

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7981

--- Comment #31 from Sidney Markowitz  ---
I've released 4.0.0-pre2 but installing that pre-release from CPAN failed,
showing that partial fix we did in bug 7986 is not good enough. I've re-opened
the bug and labeled it a 4.0.0 blocker.

This should not affect people testing 4.0.0-pre2. This only breaks some of the
spamd tests when they are run in a directory of absolute path length greater
than about 80. Almost all use cases can control that. The workaround for
installing from CPAN is anything equivalent to

$ mkdir -p /tmp/x
$ PERL_CPANM_HOME=/tmp/x cpanm -S
SIDNEY/Mail-SpamAssassin-4.0.0-pre2-TRIAL.tar.gz

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7986] spamd test fails when in directory with absolute pathname length greater than 50

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7986

Sidney Markowitz  changed:

   What|Removed |Added

 Status|RESOLVED|REOPENED
   Target Milestone|Undefined   |4.0.0
 Resolution|FIXED   |---
   Severity|normal  |blocker

--- Comment #11 from Sidney Markowitz  ---
The fix was not good enough. When I try to install 4.0.0-pre2 from CPAN on a
linux box with my home directory named /home/sidney the path length is too long
in some of the tests. We need a bit better workaround. Here are the relevant
log lines:

May 31 11:29:22.698 [468170] dbg: spamd: unix socket:
/home/sidney/.cpanm/work/1653952556.460016/Mail-SpamAssassin-4.0.0/t/log/spamd_unix_and_tcp.glHlqF/spamd.sock
May 31 11:29:22.698 [468170] dbg: spamd: [...]  Local:
/home/sidney/.cpanm/work/1653952556.460016/Mail-SpamAssassin-4.0.0/t/log/spamd_unix_and_tcp.glHlqF/spamd.sock
May 31 11:29:22.698 [468170] dbg: spamd: [...]  Type: 1
Path length (109) is longer than maximum supported length (108) and will be
truncated at /usr/lib/x86_64-linux-gnu/perl/5.30/Socket.pm line 872.
spamd: socket path was truncated at position 108
spamd: leaving stale socket at
/home/sidney/.cpanm/work/1653952556.460016/Mail-SpamAssassin-4.0.0/t/log/spamd_unix_and_tcp.glHlqF/spamd.soc
server socket setup failed, retry 1: spamd: path length for UNIX socket on
/home/sidney/.cpanm/work/1653952556.460016/Mail-SpamAssassin-4.0.0/t/log/spamd_unix_and_tcp.glHlqF/spamd.sock
exceeds system limit, exiting

-- 
You are receiving this mail because:
You are the assignee for the bug.

New Pre-Release: 4.0.0-pre2 - Testers Needed

2022-05-30 Thread Sidney Markowitz

Hello Everyone,

4.0.0 pre-release 2 is now available at
https://people.apache.org/~sidney/devel

This one has a number of fixes and improvements over 4.0.0pre1.

This is the second pre-release for the major release 4.0.0

Please test and post success or failure as soon as possible!

The draft release announcement is not yet written. It will include a
summary of changes and information for upgrading from 3.4.x.
I'll post it here if it is completed before a new release candidate.

sha256sum of archive files:

  a038fbc9b3fb43920a6849e2c089731edcdbe61af2933e58eb29dc3ac602eec6  
Mail-SpamAssassin-4.0.0-pre2.tar.bz2
  0ba8d27c916a8b63be1aa724b32a72a620852a19131ca1932412c6374e6d595f  
Mail-SpamAssassin-4.0.0-pre2.tar.gz
  71af57cd73bb1e8ae710db8cf01e893d9d27b5431a1394cb5da2eefa92659d25  
Mail-SpamAssassin-4.0.0-pre2.zip
  74db864800e5caab0747e45a551538a86179f0f15dfef6053ad9449ccaeec453  
Mail-SpamAssassin-rules-4.0.0-pre2.r1901380.tgz

sha512sum of archive files:


e17be9041edefb312221d2b85a729ab24a5fec8a6d2e352354c8fc51c60dcacb71c302b7940e09bc2a7c4a7af46ca288de201d9e84a658f3a23ecad8fac9cf38 
 Mail-SpamAssassin-4.0.0-pre2.tar.bz2


460d5ea35ac6aaf5d4caf64a345e77dd1d83eccf1e470dc43e54689e698dd8eda62fee18598b96887c6f65264b11db18eeba7f3dd299250d113c198a259f2035 
 Mail-SpamAssassin-4.0.0-pre2.tar.gz


8005f0a09938074359e1ca9fb716d9c015ca6e8dfb61cbff4713bed4ecefea2abcc753cc27b8078b0c17ce17c8210c1010c730762d61c7632afff6e09065eede 
 Mail-SpamAssassin-4.0.0-pre2.zip


198b8d58233c888395fb95635e4621e7bf403365a1feb61948f055a4522d00cfc61c974b73346ef77cbb8b603221cb30067ea039e74ba9a045e4a4825cd3f526 
 Mail-SpamAssassin-rules-4.0.0-pre2.r1901380.tgz


Fingerprints of keys for gpg signatures in the .asc files:

Install files: D8099BC79E17D7E49BC21E31FDE52F40F7D39814
Rule files:5E541DC959CB8BAC7C78DFDC4056A61A5244EC45

Regards,
Sidney Markowitz
Chair, Apache SpamAssassin PMC
sid...@apache.org


[Bug 8000] New: AMAZON_IMG_NOT_RCVD_AMZN rule producing false positives

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8000

Bug ID: 8000
   Summary: AMAZON_IMG_NOT_RCVD_AMZN rule producing false
positives
   Product: Spamassassin
   Version: 3.4.0
  Hardware: PC
OS: Linux
Status: NEW
  Severity: normal
  Priority: P2
 Component: Rules
  Assignee: dev@spamassassin.apache.org
  Reporter: de...@gerasimov.net
  Target Milestone: Undefined

Created attachment 5786
  --> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5786=edit
Example of false positive

Every legit email from amazon ordering system produces this false positive for
me. Example attached

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7999] Plugins DCC.pm, FreeMail.pm, Phishing.pm, URIDetail.pm: return undef after got_hit/rule_ready

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7999

Henrik Krohns  changed:

   What|Removed |Added

 CC||apa...@hege.li

--- Comment #3 from Henrik Krohns  ---
"superfluous" is completely correct, as this would be no blocker for 4.0.0 or
make any difference how things work.

Likely some of these can be even rewritten for better logic, as FreeMail etc is
not even async, there is no point returning undef anyway.

I'll take this just as a hint on what to work on, so this should require no
ICLA as I'm not applying it.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7981] Remaining tasks for 4.0.0 release

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7981

--- Comment #30 from Henrik Krohns  ---
(In reply to Michael Storz from comment #29)
> (In reply to Henrik Krohns from comment #27)
> > Ok I just stumbled on some eval functions returning undef wrongly..
> ...
> > All would be fixed days ago already, if Michael simply had reported which
> > plugins or functions are affected, if he already went through all this.
> > 
> 
> Henrik, you said my patches could be cosmetic at most and Sydney didn't want
> to wait for the patches and started rolling out prev2 instead. This was a
> clear sign to me that there was no interest in my patches. So why on earth
> would I rush to make the patches available? A simple question asking if I
> could upload the patches that are already done would have sufficed.

I have only vague idea of what you are working on, or what exactly you consider
problems ("chaos"), and when you might be done with them.

Then there's the problem of ICLA, which you didn't acknowledge if you are going
to sign and submit, so your patches can even be accepted.

I appreciate some of your ideas from which some good improvements and bug fixes
have resulted, but I'm simply too stressed over this whole 4.0.0 release
process that just keeps going on and on. There isn't enough stamina to wait for
some vaguely defined patches, which might or might not solve something actually
relevant. Again, performance or messy code patches can wait for later release.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7995] Add more lookups to properly leverage all Spamhaus DQS potential (and a note about DBL)

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7995

--- Comment #17 from Henrik Krohns  ---
hashbl_email_domain_alias added, couldn't think of anything needing a more
generic solution.

Committed revision 1901419.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7999] Plugins DCC.pm, FreeMail.pm, Phishing.pm, URIDetail.pm: return undef after got_hit/rule_ready

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7999

Sidney Markowitz  changed:

   What|Removed |Added

 CC||sid...@sidney.com

--- Comment #2 from Sidney Markowitz  ---
(In reply to Michael Storz from comment #1)
> Created attachment 5785 [details]

Hi Michael, could you please submit an ICLA if you are going to submit patches?
I expect that Henrik is in a better position to look at and evaluate the patch
itself, but I'm handling the bureaucratic details required by ASF of making
sure that all submitted code comes from someone who has filed an ICLA.
See https://www.apache.org/licenses/contributor-agreements.html

Thanks

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7981] Remaining tasks for 4.0.0 release

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7981

--- Comment #29 from Michael Storz  ---
(In reply to Henrik Krohns from comment #27)
> Ok I just stumbled on some eval functions returning undef wrongly..
...
> All would be fixed days ago already, if Michael simply had reported which
> plugins or functions are affected, if he already went through all this.
> 

Henrik, you said my patches could be cosmetic at most and Sydney didn't want to
wait for the patches and started rolling out prev2 instead. This was a clear
sign to me that there was no interest in my patches. So why on earth would I
rush to make the patches available? A simple question asking if I could upload
the patches that are already done would have sufficed.

But this is not my normal way of working. I usually make patches available only
when I am completely satisfied with them. And since there are still problems
with got_hit and rule_ready from my point of view, the problem with the correct
return values is not completely solved for me yet.

But I have already uploaded the patches for return value undef after calling
got_hit/rule_ready to avoid further complaints.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7999] Plugins DCC.pm, FreeMail.pm, Phishing.pm, URIDetail.pm: return undef after got_hit/rule_ready

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7999

Michael Storz  changed:

   What|Removed |Added

 CC||sa-...@lrz.de

--- Comment #1 from Michael Storz  ---
Created attachment 5785
  --> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5785=edit
return undef after call to got_hit/rule_ready

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7999] New: Plugins DCC.pm, FreeMail.pm, Phishing.pm, URIDetail.pm: return undef after got_hit/rule_ready

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7999

Bug ID: 7999
   Summary: Plugins DCC.pm, FreeMail.pm, Phishing.pm,
URIDetail.pm: return undef after got_hit/rule_ready
   Product: Spamassassin
   Version: 4.0.0
  Hardware: All
OS: All
Status: NEW
  Severity: trivial
  Priority: P2
 Component: Plugins
  Assignee: dev@spamassassin.apache.org
  Reporter: sa-...@lrz.de
  Target Milestone: Undefined

After an explicit call to got_hit or rule_ready the return value should be
undef instead of 1 or 0, otherwise another implicit call to got_hit or
rule_ready would be triggered, which is superfluous.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 6995] specify user to fall back for spamd instead of nobody

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6995

Henrik Krohns  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|NEW |RESOLVED

--- Comment #14 from Henrik Krohns  ---
No CLA, but committing as trivial, adjusted docs a bit.

Committed revision 1901416.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7981] Remaining tasks for 4.0.0 release

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7981

Loren Wilton  changed:

   What|Removed |Added

 CC|lwil...@earthlink.net   |

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7735] Meta rules need to handle missing/unrun dependencies

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735

Loren Wilton  changed:

   What|Removed |Added

 CC|lwil...@earthlink.net   |

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7987] DNSEval.pm,HashBL.pm,URILocalBL.pm: unnecessary use of rule_pending and rule_ready

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7987

Henrik Krohns  changed:

   What|Removed |Added

 Depends on||7735

--- Comment #30 from Henrik Krohns  ---
- Fix eval functions returning unintended "undef"

Committed revision 1901403.


Referenced Bugs:

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735
[Bug 7735] Meta rules need to handle missing/unrun dependencies
-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7735] Meta rules need to handle missing/unrun dependencies

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735

Henrik Krohns  changed:

   What|Removed |Added

 Blocks||7987


Referenced Bugs:

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7987
[Bug 7987] DNSEval.pm,HashBL.pm,URILocalBL.pm: unnecessary use of rule_pending
and rule_ready
-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7981] Remaining tasks for 4.0.0 release

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7981

--- Comment #28 from Sidney Markowitz  ---
(In reply to Henrik Krohns from comment #27)
> But I'm looking for all now, at least most are found with a little bit of
> extra debugging from Check.pm.

Let us know when you are done looking and I can build a pre2. Or open an issue
for it and close it when you are finished, then I'll build.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7735] Meta rules need to handle missing/unrun dependencies

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735

Henrik Krohns  changed:

   What|Removed |Added

 Status|REOPENED|RESOLVED
 Resolution|--- |FIXED

--- Comment #46 from Henrik Krohns  ---
I guess it was due to score 0 rules messing up the list. Closing this, can be
reopened if someone detects a failing test later.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7735] Meta rules need to handle missing/unrun dependencies

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735

--- Comment #45 from Sidney Markowitz  ---
(In reply to Henrik Krohns from comment #43)
> Not seeing any unrun variations after that commit. Can you make any tests
> fail?

Everything I tried passed. If you are happy with this, you can close this bug.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7998] make clean should delete rules/70_sandbox.cf and rules/72_active.cf

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7998

Sidney Markowitz  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|NEW |RESOLVED

--- Comment #1 from Sidney Markowitz  ---
trunk % svn ci -m "bug 7998 Add two files to make clean that were dropped from
distribution some time ago" Makefile.PL 
SendingMakefile.PL
Transmitting file data .done
Committing transaction...
Committed revision 1901399.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7735] Meta rules need to handle missing/unrun dependencies

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735

--- Comment #44 from Sidney Markowitz  ---
(In reply to Henrik Krohns from comment #43)
> Not seeing any unrun variations after that commit. Can you make any tests
> fail?

That fix makes a lot more sense to me. My tests looking for unrun variations
all work. I'm now running the full suite of tests to make sure nothing else has
gone wrong.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7981] Remaining tasks for 4.0.0 release

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7981

--- Comment #27 from Henrik Krohns  ---
Ok I just stumbled on some eval functions returning undef wrongly..

Like OLEVBMacro

sub check_olemacro {
  my ($self, $pms) = @_;

  return $pms->{olemacro_exists};
}

Or HTMLEval

sub html_test {
  my ($self, $pms, undef, $test) = @_;
  return $pms->{html}{$test};
}

All would be fixed days ago already, if Michael simply had reported which
plugins or functions are affected, if he already went through all this.

But I'm looking for all now, atleast most are found with a little bit of extra
debugging from Check.pm.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7735] Meta rules need to handle missing/unrun dependencies

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735

--- Comment #43 from Henrik Krohns  ---
Not seeing any unrun variations after that commit. Can you make any tests fail?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7735] Meta rules need to handle missing/unrun dependencies

2022-05-30 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7735

--- Comment #42 from Henrik Krohns  ---

- Disabled (score 0) metas no longer added to meta_pending
- Unrun debug reporting should now be foolproof

Committed revision 1901397.

-- 
You are receiving this mail because:
You are the assignee for the bug.