[
https://issues.apache.org/jira/browse/SUREFIRE-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922338#comment-16922338
]
Tibor Digana edited comment on SUREFIRE-1691 at 9/4/19 9:54 AM:
----------------------------------------------------------------
[~mpkorstanje]
A fix is not worth the spent time because we want to destroy the
ThreadLocal-based solution and we want the StatelessXmlReporter been able to
resolve parallel events received from the tests. This solution in reporter is
needed in JUni5 because the tests with JUnit5 provider have problem to separate
parallel logs. This way we would solve two bugs by one fix. The fix was planned
for M4 but honestly it is so difficult that we have to postpone it to M5.
was (Author: tibor17):
[~mpkorstanje]
A fix is not worth the spent time because we want to destroy the
ThreadLocal-based solution and we want the StatelessXmlReporter been able to
resolve parallel events received from the tests. This solution in reporter is
needed in JUni5 because the tests with JUnit5 provider have problem to separate
logs from parallel logs. This way we would solve two bugs by one fix. The fix
was planned for M4 but honestly it is so difficult that we have to postpone it
to M5.
> surefire-junit47 reports results against incorrect test
> -------------------------------------------------------
>
> Key: SUREFIRE-1691
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1691
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.x support
> Affects Versions: 3.0.0-M3
> Reporter: M.P. Korstanje
> Priority: Major
>
> This is a follow up on:
> - https://issues.apache.org/jira/browse/SUREFIRE-1156
> - [https://github.com/cucumber/cucumber-jvm/issues/865]
> The latest reproducer can be found here:
> -
> [https://github.com/mpkorstanje/surefire-test/tree/07d0ee98ea424b38a3c548981a875dde1ef4f2c2]
> In short Cucumber has a hierarchy of tests that looks at follows:
> {noformat}
> Class annotated with @RunWith(Cucumber.class)
> |- Feature File 1
> | |- Scenario 1a
> | |- Scenario 1b
> |- Feature File 2
> | |- Scenario 2a
> | |- Scenario 2b
> {noformat}
> Surefire tries to group output in tests sets, either per class or per test
> suite. As Cucumber currently only emits test started/finished events for
> Scenarios sure fire can not group scenarios into a test set. They are instead
> grouped with the previous test set. However even after adding test
> started/finished events for features, sure fire groups all results in a
> single test set.
> I believe this is caused by an error in
> {{NonConcurrentRunListener.describesNewTestSet}}.
> {code:java}
> private boolean describesNewTestSet( Description description )
> {
> if ( currentTestSetDescription != null )
> {
> if ( null != description.getTestClass() )
> {
> return !description.getTestClass().equals(
> currentTestSetDescription.getTestClass() );
> }
> else if ( description.isSuite() )
> {
> return description.getChildren().equals(
> currentTestSetDescription.getChildren() );
> }
> return false;
> }
> return true;
> }
> {code}
> The value of {{description.getChildren().equals(
> currentTestSetDescription.getChildren() );}} should be negated so so
> different test suits are properly recognized as test sets.
> Fixing this locally works and can be reproduced by:
> 1. Negating the line and building sure fire locally
> 2. Building [https://github.com/cucumber/cucumber-jvm/pull/1765] with `mvn
> clean install -DskipTests`
> 3. Running `mvn clean test -Pjunit47` on
> [https://github.com/mpkorstanje/surefire-test]
> It does however does result in another unwanted result: A feature file with
> two scenarios is now reported as having 3 results. One for each scenario and
> one for the feature.
> {noformat}
> -------------------------------------------------------------------------------
> Test set: Some Feature 1
> -------------------------------------------------------------------------------
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.112 s - in
> Some_Feature_1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
