[
https://issues.apache.org/jira/browse/MATH-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708483#comment-17708483
]
Alex Herbert commented on MATH-1659:
------------------------------------
This is actually a typical build output. All the failures are in the optimizer
tests. These are rerun as they are allowed to fail. The tests are marked as
flakes in the final build report. Your attached trace actually passed all the
tests within the allowed number of failures.
The optimizer tests were created using a set of hard to solve functions with
many local minima. These would be difficult for any optimizer. The optimizer
start points are randomly chosen, hence often the solution is not the optimal
one.
Ideally the output should be suppressed so that errors are only shown when all
repeats fail. The tests could be rewritten to use custom set-up and assertion
operations within a repeat loop with a given number of retries. Currently the
tests use the retries functionality of the surefire plugin (see [Maven Surefire
plugin: re-run failing
tests|https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html]).
One alternative is to use fixed random seeds to initialize the optimizers. But
this can create an unrealistic test which can output a pass although pass is a
minority fraction of all results (pass or fail). This minority pass fraction is
what we have for a lot of the optimizer tests as they require many repeats to
observe a pass. So here at least we are aware of the flakyness of tests, just
the report is very verbose. I cannot see an option within surefire to quieten
down the console output for the build.
> some test failures on legacy 4.0 beta1
> --------------------------------------
>
> Key: MATH-1659
> URL: https://issues.apache.org/jira/browse/MATH-1659
> Project: Commons Math
> Issue Type: Test
> Components: legacy
> Affects Versions: 4.0-beta1
> Environment: Dev configuration where the problem occurs is
> * Ubuntu 22.04,
> * Maven 3.6.3
> * Eclipse 4.23,
> * java-18-openJdk-amd64 with JavaSE-1.8 compliance
> Reporter: François Laferrière
> Priority: Major
> Attachments: maven.test.trace.txt
>
>
> When running Junit tests on ACM 4.0-snapshot I get a lot of failures. Mostly
> failures are associated to
> * roundoff error beyond the tolerance of individual test.
> * max number of function evalution exceded
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
