[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[
https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475322#comment-16475322
]
Heiko Wentzke commented on SUREFIRE-1480:
-
Updated to Surefire v2.21.0 - still the same result in Jenkins :(
> parallel tests may produce invalid .xml report
> --
>
> Key: SUREFIRE-1480
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1480
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin,
> Maven Surefire Plugin
>Affects Versions: 2.20.1
>Reporter: Mark Lehky
>Priority: Major
> Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt
>
>
> Relevant software:
> * Jenkins 2.108
> * Maven 3.??
> * JUnit 4.12
> * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)
> I have a testsuite (one JUnit class) that contains multiple tests (multiple
> JUnit methods), which are all run in parallel. Some of the tests may be
> ignore using JUnit {{Assume}}.
> On occasion (not 100% reproducible), the resulting report will contain an
> entry like:
> {noformat}
> < message="Skip test!">
> {noformat}
> The correct entry, as is produced most of the time, should be:
> {noformat}
>
> {noformat}
> The invalid formatted XML, when run in Jenkins, results in the test being
> flagged as failed, and Jenkins simply has the message:
> "TEST-xml.[failed-to-read]" (the dots are replaced with the correct
> filename!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[
https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436898#comment-16436898
]
Heiko Wentzke commented on SUREFIRE-1480:
-
For me, nothing has changed, I still have the same issues randomly happening to
some test classes
> parallel tests may produce invalid .xml report
> --
>
> Key: SUREFIRE-1480
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1480
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin,
> Maven Surefire Plugin
>Affects Versions: 2.20.1
>Reporter: Mark Lehky
>Priority: Major
> Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt
>
>
> Relevant software:
> * Jenkins 2.108
> * Maven 3.??
> * JUnit 4.12
> * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)
> I have a testsuite (one JUnit class) that contains multiple tests (multiple
> JUnit methods), which are all run in parallel. Some of the tests may be
> ignore using JUnit {{Assume}}.
> On occasion (not 100% reproducible), the resulting report will contain an
> entry like:
> {noformat}
> < message="Skip test!">
> {noformat}
> The correct entry, as is produced most of the time, should be:
> {noformat}
>
> {noformat}
> The invalid formatted XML, when run in Jenkins, results in the test being
> flagged as failed, and Jenkins simply has the message:
> "TEST-xml.[failed-to-read]" (the dots are replaced with the correct
> filename!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[
https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435015#comment-16435015
]
Heiko Wentzke commented on SUREFIRE-1480:
-
I will do some testing today with an updated version of failsafe and let you
know!
> parallel tests may produce invalid .xml report
> --
>
> Key: SUREFIRE-1480
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1480
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin,
> Maven Surefire Plugin
>Affects Versions: 2.20.1
>Reporter: Mark Lehky
>Priority: Major
> Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt
>
>
> Relevant software:
> * Jenkins 2.108
> * Maven 3.??
> * JUnit 4.12
> * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)
> I have a testsuite (one JUnit class) that contains multiple tests (multiple
> JUnit methods), which are all run in parallel. Some of the tests may be
> ignore using JUnit {{Assume}}.
> On occasion (not 100% reproducible), the resulting report will contain an
> entry like:
> {noformat}
> < message="Skip test!">
> {noformat}
> The correct entry, as is produced most of the time, should be:
> {noformat}
>
> {noformat}
> The invalid formatted XML, when run in Jenkins, results in the test being
> flagged as failed, and Jenkins simply has the message:
> "TEST-xml.[failed-to-read]" (the dots are replaced with the correct
> filename!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[
https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heiko Wentzke updated SUREFIRE-1480:
Attachment: Stacktrace_failedTest.txt
FailedXMLReport.txt
> parallel tests may produce invalid .xml report
> --
>
> Key: SUREFIRE-1480
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1480
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Failsafe Plugin, Maven Surefire Plugin
>Affects Versions: 2.20.1
>Reporter: Mark Lehky
>Priority: Major
> Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt
>
>
> Relevant software:
> * Jenkins 2.108
> * Maven 3.??
> * JUnit 4.12
> * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)
> I have a testsuite (one JUnit class) that contains multiple tests (multiple
> JUnit methods), which are all run in parallel. Some of the tests may be
> ignore using JUnit {{Assume}}.
> On occasion (not 100% reproducible), the resulting report will contain an
> entry like:
> {noformat}
> < message="Skip test!">
> {noformat}
> The correct entry, as is produced most of the time, should be:
> {noformat}
>
> {noformat}
> The invalid formatted XML, when run in Jenkins, results in the test being
> flagged as failed, and Jenkins simply has the message:
> "TEST-xml.[failed-to-read]" (the dots are replaced with the correct
> filename!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[
https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383349#comment-16383349
]
Heiko Wentzke commented on SUREFIRE-1480:
-
Hello,
I am experiencing the same issue with my test configuration!
Jenkins 2.50
Maven 3.3.9
Surefire Plugin 2.20.1
Junit 4.12
Running tests in parallel with rerunFailed set to 1
When I execute my builds in Jenkins, some tests randomly fail with an invalid
report xml.
Tests are getting flagged as
{code:java}
TEST-com.company.tests.testName.xml.[failed-to-read]
{code}
For my builds, this mostly happens with one specific testclass at random, but
it also affects other classes randomly too.
When I check the invalid XML file, I can see that it only contains the
stacktrace of one test instead of the normal XML report data.
see attached
[^FailedXMLReport.txt] [^Stacktrace_failedTest.txt]
> parallel tests may produce invalid .xml report
> --
>
> Key: SUREFIRE-1480
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1480
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Failsafe Plugin, Maven Surefire Plugin
>Affects Versions: 2.20.1
>Reporter: Mark Lehky
>Priority: Major
> Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt
>
>
> Relevant software:
> * Jenkins 2.108
> * Maven 3.??
> * JUnit 4.12
> * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)
> I have a testsuite (one JUnit class) that contains multiple tests (multiple
> JUnit methods), which are all run in parallel. Some of the tests may be
> ignore using JUnit {{Assume}}.
> On occasion (not 100% reproducible), the resulting report will contain an
> entry like:
> {noformat}
> < message="Skip test!">
> {noformat}
> The correct entry, as is produced most of the time, should be:
> {noformat}
>
> {noformat}
> The invalid formatted XML, when run in Jenkins, results in the test being
> flagged as failed, and Jenkins simply has the message:
> "TEST-xml.[failed-to-read]" (the dots are replaced with the correct
> filename!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
