Jens Reimann created SUREFIRE-1799:
--------------------------------------
Summary: Exception thrown in @MethodSource method results in "no
tests"
Key: SUREFIRE-1799
URL: https://issues.apache.org/jira/browse/SUREFIRE-1799
Project: Maven Surefire
Issue Type: Bug
Components: JUnit 5.x support
Affects Versions: 3.0.0-M4
Reporter: Jens Reimann
Running the following code:
{code:java}
import java.util.List;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.TestInstance.Lifecycle;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
@TestInstance(Lifecycle.PER_CLASS)
public class MethodSourceErrorTest {
public List<String> getValues() {
throw new RuntimeException("Foo"); // fails
// return Arrays.asList("a"); // works
}
@ParameterizedTest
@MethodSource("getValues")
public void test1(String value) {
System.out.println(value);
}
}
{code}
Results in:
{code}
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running foo.MethodSourceErrorTest
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 s
- in foo.MethodSourceErrorTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
{code}
My expectation would be that:
* JUnit reports an error for the tests using the method source
* The build fails
Additional information:
* It fails the same way, being a static method
Used versions (Jupiter/Vintage/Platform): 5.6.2, 1.6.2
I raised an issue with JUnit 5
(https://github.com/junit-team/junit5/issues/2319), however they claim that it
is an issue with the tool running the test (Maven Surefire in this case).
And indeed the TestExecutionListener receives an exception:
{code:java}
public class MyTestExecutionListener implements TestExecutionListener {
public MyTestExecutionListener() {
System.out.println("Here");
}
@Override
public void executionFinished(TestIdentifier testIdentifier,
TestExecutionResult testExecutionResult) {
System.out.format("finished - id: %s, result: %s%n", testIdentifier,
testExecutionResult);
}
}
{code}
Shows the exception as:
{code}
[INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @ bar ---
[WARNING] useSystemClassloader setting has no effect when not forking
Here
[INFO] Running foo.MethodSourceErrorTest
finished - id: TestIdentifier [uniqueId =
'[engine:junit-jupiter]/[class:foo.MethodSourceErrorTest]/[test-template:test1(java.lang.String)]',
parentId = '[engine:junit-jupiter]/[class:foo.MethodSourceErrorTest]',
displayName = 'test1(String)', legacyReportingName = 'test1(String)', source =
MethodSource [className = 'foo.MethodSourceErrorTest', methodName = 'test1',
methodParameterTypes = 'java.lang.String'], tags = [], type = CONTAINER],
result: TestExecutionResult [status = FAILED, throwable =
java.lang.RuntimeException: Foo]
finished - id: TestIdentifier [uniqueId =
'[engine:junit-jupiter]/[class:foo.MethodSourceErrorTest]', parentId =
'[engine:junit-jupiter]', displayName = 'MethodSourceErrorTest',
legacyReportingName = 'foo.MethodSourceErrorTest', source = ClassSource
[className = 'foo.MethodSourceErrorTest', filePosition = null], tags = [], type
= CONTAINER], result: TestExecutionResult [status = SUCCESSFUL, throwable =
null]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.013 s
- in foo.MethodSourceErrorTest
finished - id: TestIdentifier [uniqueId = '[engine:junit-jupiter]', parentId =
null, displayName = 'JUnit Jupiter', legacyReportingName = 'JUnit Jupiter',
source = null, tags = [], type = CONTAINER], result: TestExecutionResult
[status = SUCCESSFUL, throwable = null]
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)