[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[ https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475322#comment-16475322 ] Heiko Wentzke commented on SUREFIRE-1480: - Updated to Surefire v2.21.0 - still the same result in Jenkins :( > parallel tests may produce invalid .xml report > -- > > Key: SUREFIRE-1480 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1480 > Project: Maven Surefire > Issue Type: Bug > Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin, > Maven Surefire Plugin >Affects Versions: 2.20.1 >Reporter: Mark Lehky >Priority: Major > Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt > > > Relevant software: > * Jenkins 2.108 > * Maven 3.?? > * JUnit 4.12 > * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well) > I have a testsuite (one JUnit class) that contains multiple tests (multiple > JUnit methods), which are all run in parallel. Some of the tests may be > ignore using JUnit {{Assume}}. > On occasion (not 100% reproducible), the resulting report will contain an > entry like: > {noformat} > < message="Skip test!"> > {noformat} > The correct entry, as is produced most of the time, should be: > {noformat} > > {noformat} > The invalid formatted XML, when run in Jenkins, results in the test being > flagged as failed, and Jenkins simply has the message: > "TEST-xml.[failed-to-read]" (the dots are replaced with the correct > filename!). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[ https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436898#comment-16436898 ] Heiko Wentzke commented on SUREFIRE-1480: - For me, nothing has changed, I still have the same issues randomly happening to some test classes > parallel tests may produce invalid .xml report > -- > > Key: SUREFIRE-1480 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1480 > Project: Maven Surefire > Issue Type: Bug > Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin, > Maven Surefire Plugin >Affects Versions: 2.20.1 >Reporter: Mark Lehky >Priority: Major > Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt > > > Relevant software: > * Jenkins 2.108 > * Maven 3.?? > * JUnit 4.12 > * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well) > I have a testsuite (one JUnit class) that contains multiple tests (multiple > JUnit methods), which are all run in parallel. Some of the tests may be > ignore using JUnit {{Assume}}. > On occasion (not 100% reproducible), the resulting report will contain an > entry like: > {noformat} > < message="Skip test!"> > {noformat} > The correct entry, as is produced most of the time, should be: > {noformat} > > {noformat} > The invalid formatted XML, when run in Jenkins, results in the test being > flagged as failed, and Jenkins simply has the message: > "TEST-xml.[failed-to-read]" (the dots are replaced with the correct > filename!). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[ https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435015#comment-16435015 ] Heiko Wentzke commented on SUREFIRE-1480: - I will do some testing today with an updated version of failsafe and let you know! > parallel tests may produce invalid .xml report > -- > > Key: SUREFIRE-1480 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1480 > Project: Maven Surefire > Issue Type: Bug > Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin, > Maven Surefire Plugin >Affects Versions: 2.20.1 >Reporter: Mark Lehky >Priority: Major > Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt > > > Relevant software: > * Jenkins 2.108 > * Maven 3.?? > * JUnit 4.12 > * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well) > I have a testsuite (one JUnit class) that contains multiple tests (multiple > JUnit methods), which are all run in parallel. Some of the tests may be > ignore using JUnit {{Assume}}. > On occasion (not 100% reproducible), the resulting report will contain an > entry like: > {noformat} > < message="Skip test!"> > {noformat} > The correct entry, as is produced most of the time, should be: > {noformat} > > {noformat} > The invalid formatted XML, when run in Jenkins, results in the test being > flagged as failed, and Jenkins simply has the message: > "TEST-xml.[failed-to-read]" (the dots are replaced with the correct > filename!). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[ https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heiko Wentzke updated SUREFIRE-1480: Attachment: Stacktrace_failedTest.txt FailedXMLReport.txt > parallel tests may produce invalid .xml report > -- > > Key: SUREFIRE-1480 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1480 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Failsafe Plugin, Maven Surefire Plugin >Affects Versions: 2.20.1 >Reporter: Mark Lehky >Priority: Major > Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt > > > Relevant software: > * Jenkins 2.108 > * Maven 3.?? > * JUnit 4.12 > * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well) > I have a testsuite (one JUnit class) that contains multiple tests (multiple > JUnit methods), which are all run in parallel. Some of the tests may be > ignore using JUnit {{Assume}}. > On occasion (not 100% reproducible), the resulting report will contain an > entry like: > {noformat} > < message="Skip test!"> > {noformat} > The correct entry, as is produced most of the time, should be: > {noformat} > > {noformat} > The invalid formatted XML, when run in Jenkins, results in the test being > flagged as failed, and Jenkins simply has the message: > "TEST-xml.[failed-to-read]" (the dots are replaced with the correct > filename!). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[ https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383349#comment-16383349 ] Heiko Wentzke commented on SUREFIRE-1480: - Hello, I am experiencing the same issue with my test configuration! Jenkins 2.50 Maven 3.3.9 Surefire Plugin 2.20.1 Junit 4.12 Running tests in parallel with rerunFailed set to 1 When I execute my builds in Jenkins, some tests randomly fail with an invalid report xml. Tests are getting flagged as {code:java} TEST-com.company.tests.testName.xml.[failed-to-read] {code} For my builds, this mostly happens with one specific testclass at random, but it also affects other classes randomly too. When I check the invalid XML file, I can see that it only contains the stacktrace of one test instead of the normal XML report data. see attached [^FailedXMLReport.txt] [^Stacktrace_failedTest.txt] > parallel tests may produce invalid .xml report > -- > > Key: SUREFIRE-1480 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1480 > Project: Maven Surefire > Issue Type: Bug > Components: Maven Failsafe Plugin, Maven Surefire Plugin >Affects Versions: 2.20.1 >Reporter: Mark Lehky >Priority: Major > Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt > > > Relevant software: > * Jenkins 2.108 > * Maven 3.?? > * JUnit 4.12 > * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well) > I have a testsuite (one JUnit class) that contains multiple tests (multiple > JUnit methods), which are all run in parallel. Some of the tests may be > ignore using JUnit {{Assume}}. > On occasion (not 100% reproducible), the resulting report will contain an > entry like: > {noformat} > < message="Skip test!"> > {noformat} > The correct entry, as is produced most of the time, should be: > {noformat} > > {noformat} > The invalid formatted XML, when run in Jenkins, results in the test being > flagged as failed, and Jenkins simply has the message: > "TEST-xml.[failed-to-read]" (the dots are replaced with the correct > filename!). -- This message was sent by Atlassian JIRA (v7.6.3#76005)