https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8201
Bug ID: 8201
Summary: Make some tests that fail occasionally more robust
Product: Spamassassin
Version: SVN Trunk (Latest Devel Version)
Hardware: All
OS: All
Status: NEW
Severity: minor
Priority: P2
Component: Regression Tests
Assignee: dev@spamassassin.apache.org
Reporter: sid...@sidney.com
Target Milestone: Undefined
Created attachment 5920
--> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5920&action=edit
proposed patch
There are four tests that are written to iterate $iterations times, with
explanation in the comment that it is to allow for random natural failures. I'm
seeing such occasional failures showing up on CPAN testing machines running
Windows with versions of Strawberry perl 5.24 and older
basic_meta2.t dnsbl.t hashbl.t uribl.t
The problem with the current implementation is that a failure in any of the
iterations is considered a test failure.
I'm fixing this by using a feature of ok_all_patterns that I see in the code
even though no existing tests make use of it: If passed an argument of true,
ok_all_patterns will output appropriate warnings of pattern failures and return
0, but will not ok() or not_ok() the pattern as a test. That lets me use
ok_all_patterns to count the number of iterations in which everything passes
and only fail the test if it fails every iteration.
I'm attaching a proposed patch to the four test files
--
You are receiving this mail because:
You are the assignee for the bug.
