[
https://issues.apache.org/jira/browse/SUREFIRE-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17346467#comment-17346467
]
Andreas Pabst edited comment on SUREFIRE-1914 at 5/17/21, 10:36 PM:
--------------------------------------------------------------------
PR has been modified, so that there is no change to existing ITs. Only an
additional test project and IT is added.
Note that the pre-existing IT
JUnitPlatformEnginesIT#testJupiterEngineWithDisplayNames mentioned above runs
somewhat counter to my expectations.
It explicitly expects a testcase name that omits the method signature in the
XML - at least for the one parameterized test in BasicJupiterTest. The relevant
asserts look like this:
{code:java}
.assertContainsText( "<testcase name=\"0 + 1 = 1\" "
+
"classname=\"junitplatformenginejupiter.BasicJupiterTest\"" )
{code}
As mentioned, I would rather expect something like
{code:java}
.assertContainsText( "<testcase name=\"add(int, int, int)[1] 0
+ 1 = 1\" "
+
"classname=\"junitplatformenginejupiter.BasicJupiterTest\"" )
{code}
With regards to the implementation in RunListenerAdapter, [~srdo] has posted a
first draft of what would need to be changed about it in this comment
was (Author: andpab):
PR has been modified, so that there is no change to existing ITs. Only an
additional test project and IT is added.
Note that the pre-existing IT
JUnitPlatformEnginesIT#testJupiterEngineWithDisplayNames mentioned above runs
somewhat counter to my expectations.
It explicitly expects a testcase name that omits the method signature in the
XML - at least for the one parameterized test in BasicJupiterTest. The relevant
asserts look like this:
{code:java}
.assertContainsText( "<testcase name=\"0 + 1 = 1\" "
+
"classname=\"junitplatformenginejupiter.BasicJupiterTest\"" )
{code}
With regards to the implementation in RunListenerAdapter, [~srdo] has posted a
first draft of what would need to be changed about it in [this
comment|https://issues.apache.org/jira/browse/SUREFIRE-1546?focusedCommentId=16817626&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16817626]
> XML report omits method signature / display name of Junit 5 parameterized
> tests if testset reporter is configured to use phrased naming
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SUREFIRE-1914
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1914
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support, xml generation
> Affects Versions: 3.0.0-M5
> Reporter: Andreas Pabst
> Priority: Major
>
> h3. Description of the issue
> Given a test class with two parameterized tests as follows:
> {code:java}
> @ParameterizedTest
> @ValueSource(strings = {"a", "b"})
> void test1(String param) {
> // test code
> }
> @ParameterizedTest
> @ValueSource(strings = {"a", "b"})
> void test2(String param) {
> // test code
> }
> {code}
> and a surefire configuration that includes the following:
> {code:xml}
> <statelessTestsetReporter
> implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
> <version>3.0</version>
> <usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
> <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
> <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
> </statelessTestsetReporter>
> {code}
> the XML report will look like this:
> {code:xml}
> <testcase name="[1] a" classname="mypackage.MyClass" time="0.001" />
> <testcase name="[2] b" classname="mypackage.MyClass" time="0.001" />
> <testcase name="[1] a" classname="mypackage.MyClass" time="0.001" />
> <testcase name="[2] b" classname="mypackage.MyClass" time="0.001" />
> {code}
> Note: The test method name/signature is not included in the name attribute.
> I would expect something more like this:
> {code:xml}
> <testcase name="test1(String) [1] a" classname="mypackage.MyClass"
> time="0.001" />
> <testcase name="test1(String) [2] b" classname="mypackage.MyClass"
> time="0.001" />
> <testcase name="test2(String) [1] a" classname="mypackage.MyClass"
> time="0.001" />
> <testcase name="test2(String) [2] b" classname="mypackage.MyClass"
> time="0.001" />
> {code}
> h3. Some context on why this is bad
> Omitting the test method name makes it impossible to differentiate individual
> test cases in the XML report if there are multiple
> {{@ParameterizedTest}}-annotated test cases in the same class that use the
> same parameters.
> Any software that parses the surefire XML reports will be misled into
> thinking these are multiple executions of the same test.
> h3. Variant with {{@DisplayName}} usage
> There is a variant of this issue: If the {{@ParameterizedTest}} has a
> {{@DisplayName}} annotation, whatever was chosen as a display name is not
> included in the XML report either - unless it is explicitly referenced in the
> name attribute of the {{@ParameterizedTest}} like so:
> {{@ParameterizedTest(name = "\{displayName} ...")}}.
> h3. Solution ideas
> This issue has already been brought up during the discussion of SUREFIRE-1546.
> In that discussion [~srdo] and [~marcphilipp] have described an approach how
> CONTAINER-type TestIdentifiers could be handled properly, but that particular
> problem seems to have been deferred and has seemingly not been picked up
> again since.
> h3. Workaround
> A workaround is to explicitly reference the displayName or include some other
> unique string in the name attribute of each {{@ParameterizedTest}} as
> mentioned above. In projects with a large legacy testbase that might not be
> feasible though, so this issue might prevent the adoption of the feature
> introduced in SUREFIRE-1546.
> I'm submitting this as a bug because to me the current behaviour of surefire
> is at least unexpected, but it could also be seen as a request for
> improvement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
