[ 
https://issues.apache.org/jira/browse/ARTEMIS-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robbie Gemmell updated ARTEMIS-4831:
------------------------------------
    Description: 
Currently, the artemis build explicitly configures maven/surefire to ignore 
test failures most of the time, both out of the box, and when running most of 
the test profiles. The artemis build _only_ configures maven/surefire to its 
usual standard behaviour (to fail the whole build at the module which first 
failed a test) for the 'fast-tests' profile used in the PR test runs.

This is horrible out of the box behaviour because if people dont know this, as 
many wont and almost noone would assume, then they can entirely miss the fact 
that there were test failures because they are buried by subsequent test 
output, and the end command result is then success with a nice banner from 
Maven indicating BUILD SUCCESS. Most folks not using something like Jenkins 
with the JUnit test results processing, or running the aggregate surefire 
report after testing and looking, is likely to miss failures they have 
introduced. I have seen this happen in a few PRs just recently.

Surefire already has a dedicated property to give this behaviour, easily 
configurable via the pom or CLI: {_}maven.test.failure.ignore{_}. The default 
artemis build behaviour should adopt the default surefire behaviour everyone is 
already used to and almost certainly expects it already uses. Anyone who does 
specifically wish to have the ignoring behaviour to run all modules can 
configure maven/surefire when running it, e.g:
{code:java}
mvn test -Dmaven.test.failure.ignore{code}
 

 

  was:
Currently, the artemis build explicitly configures maven/surefire out of the 
box to ignore test failures most of the time, both out of the box, and when 
running most of the test profiles. The artemis build _only_ configures 
maven/surefire it to its usual standard behaviour (to fail the whole build at 
the module which first failed a test) for the 'fast-tests' profile used in the 
PR test runs.

This is horrible out of the box behaviour because if people dont know this, as 
many wont and almost noone would assume, then they can entirely miss the fact 
that there were test failures because they are buried by subsequent test 
output, and the end command result is then success with a nice banner from 
Maven indicating BUILD SUCCESS. Most folks not using something like Jenkins 
with the JUnit test results processing, or running the aggregate surefire 
report after testing and looking, is likely to miss failures they have 
introduced. I have seen this happen in a few PRs just recently.

Surefire already has a dedicated property to give this behaviour, easily 
configurable via the pom or CLI: {_}maven.test.failure.ignore{_}. The default 
artemis build behaviour should adopt the default surefire behaviour everyone is 
already used to and almost certainly expects it already uses. Anyone who does 
specfically wish to have the prior behaviour can configure maven/surefire when 
running it, e.g:
{code:java}
mvn test -Dmaven.test.failure.ignore{code}
 

 


> consistently use surefire default behaviour around test failure
> ---------------------------------------------------------------
>
>                 Key: ARTEMIS-4831
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-4831
>             Project: ActiveMQ Artemis
>          Issue Type: Task
>    Affects Versions: 2.35.0
>            Reporter: Robbie Gemmell
>            Assignee: Robbie Gemmell
>            Priority: Major
>             Fix For: 2.36.0
>
>
> Currently, the artemis build explicitly configures maven/surefire to ignore 
> test failures most of the time, both out of the box, and when running most of 
> the test profiles. The artemis build _only_ configures maven/surefire to its 
> usual standard behaviour (to fail the whole build at the module which first 
> failed a test) for the 'fast-tests' profile used in the PR test runs.
> This is horrible out of the box behaviour because if people dont know this, 
> as many wont and almost noone would assume, then they can entirely miss the 
> fact that there were test failures because they are buried by subsequent test 
> output, and the end command result is then success with a nice banner from 
> Maven indicating BUILD SUCCESS. Most folks not using something like Jenkins 
> with the JUnit test results processing, or running the aggregate surefire 
> report after testing and looking, is likely to miss failures they have 
> introduced. I have seen this happen in a few PRs just recently.
> Surefire already has a dedicated property to give this behaviour, easily 
> configurable via the pom or CLI: {_}maven.test.failure.ignore{_}. The default 
> artemis build behaviour should adopt the default surefire behaviour everyone 
> is already used to and almost certainly expects it already uses. Anyone who 
> does specifically wish to have the ignoring behaviour to run all modules can 
> configure maven/surefire when running it, e.g:
> {code:java}
> mvn test -Dmaven.test.failure.ignore{code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information, visit: https://activemq.apache.org/contact


Reply via email to