[
https://issues.apache.org/jira/browse/ARTEMIS-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robbie Gemmell updated ARTEMIS-4831:
------------------------------------
Description:
Currently, the artemis build explicitly configures maven/surefire to ignore
test failures most of the time, both out of the box, and when running most of
the test profiles. The artemis build _only_ configures maven/surefire to its
usual standard behaviour (to fail the whole build at the module which first
failed a test) for the 'fast-tests' profile used in the PR test runs.
This is horrible out of the box behaviour because if people dont know this, as
many wont and almost noone would assume, then they can entirely miss the fact
that there were test failures because they are buried by subsequent test
output, and the end command result is then success with a nice banner from
Maven indicating BUILD SUCCESS. Most folks not using something like Jenkins
with the JUnit test results processing, or running the aggregate surefire
report after testing and looking, is likely to miss failures they have
introduced. I have seen this happen in a few PRs just recently.
Surefire already has a dedicated property to give this behaviour, easily
configurable via the pom or CLI: {_}maven.test.failure.ignore{_}. The default
artemis build behaviour should adopt the default surefire behaviour everyone is
already used to and almost certainly expects it already uses. Anyone who does
specifically wish to have the ignoring behaviour to run all modules can
configure maven/surefire when running it, e.g:
{code:java}
mvn test -Dmaven.test.failure.ignore{code}
was:
Currently, the artemis build explicitly configures maven/surefire out of the
box to ignore test failures most of the time, both out of the box, and when
running most of the test profiles. The artemis build _only_ configures
maven/surefire it to its usual standard behaviour (to fail the whole build at
the module which first failed a test) for the 'fast-tests' profile used in the
PR test runs.
This is horrible out of the box behaviour because if people dont know this, as
many wont and almost noone would assume, then they can entirely miss the fact
that there were test failures because they are buried by subsequent test
output, and the end command result is then success with a nice banner from
Maven indicating BUILD SUCCESS. Most folks not using something like Jenkins
with the JUnit test results processing, or running the aggregate surefire
report after testing and looking, is likely to miss failures they have
introduced. I have seen this happen in a few PRs just recently.
Surefire already has a dedicated property to give this behaviour, easily
configurable via the pom or CLI: {_}maven.test.failure.ignore{_}. The default
artemis build behaviour should adopt the default surefire behaviour everyone is
already used to and almost certainly expects it already uses. Anyone who does
specfically wish to have the prior behaviour can configure maven/surefire when
running it, e.g:
{code:java}
mvn test -Dmaven.test.failure.ignore{code}
> consistently use surefire default behaviour around test failure
> ---------------------------------------------------------------
>
> Key: ARTEMIS-4831
> URL: https://issues.apache.org/jira/browse/ARTEMIS-4831
> Project: ActiveMQ Artemis
> Issue Type: Task
> Affects Versions: 2.35.0
> Reporter: Robbie Gemmell
> Assignee: Robbie Gemmell
> Priority: Major
> Fix For: 2.36.0
>
>
> Currently, the artemis build explicitly configures maven/surefire to ignore
> test failures most of the time, both out of the box, and when running most of
> the test profiles. The artemis build _only_ configures maven/surefire to its
> usual standard behaviour (to fail the whole build at the module which first
> failed a test) for the 'fast-tests' profile used in the PR test runs.
> This is horrible out of the box behaviour because if people dont know this,
> as many wont and almost noone would assume, then they can entirely miss the
> fact that there were test failures because they are buried by subsequent test
> output, and the end command result is then success with a nice banner from
> Maven indicating BUILD SUCCESS. Most folks not using something like Jenkins
> with the JUnit test results processing, or running the aggregate surefire
> report after testing and looking, is likely to miss failures they have
> introduced. I have seen this happen in a few PRs just recently.
> Surefire already has a dedicated property to give this behaviour, easily
> configurable via the pom or CLI: {_}maven.test.failure.ignore{_}. The default
> artemis build behaviour should adopt the default surefire behaviour everyone
> is already used to and almost certainly expects it already uses. Anyone who
> does specifically wish to have the ignoring behaviour to run all modules can
> configure maven/surefire when running it, e.g:
> {code:java}
> mvn test -Dmaven.test.failure.ignore{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information, visit: https://activemq.apache.org/contact