I believe what is happening is that when you achieve for the second time with tests of the same name, they replace the other tests. This is described in https://issues.jenkins-ci.org/browse/JENKINS-27395
One workaround is if you modify the .xml files so that the test names are different. There's a Pipeline script function in the linked ticket that can help with that https://issues.jenkins-ci.org/browse/JENKINS-27395?focusedCommentId=256459&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-256459 This is not ideal though - we should really fix this. On Friday, August 19, 2016 at 3:33:52 AM UTC+10, Joshua Harlow wrote: > > It also gets interesting that if I have 2 runs (pre-patch unit testing and > post-patch unittest) but both have tests with the same name in them, > > Unsure exactly how that will render :-P > > I'll see what I can find out, other option I guess is to just publish a > bunch of html (and not use the results renderer). > > -Josh > > On Wednesday, August 17, 2016 at 10:19:13 PM UTC-7, Joshua Harlow wrote: >> >> Do I need to just give each capture a unique filename? >> >> I was trying this earlier and it appears giving them the same name only >> caused the first to show up on the UI :( >> >> Perhaps though I was doing something wrong... >> >> On Wednesday, August 17, 2016 at 9:24:37 PM UTC-7, James Dumay wrote: >>> >>> Joshua, I don't think there is anything stopping you from recording the >>> test results in the Pipeline you have described. Jenkins will capture them >>> wherever you ask it to with the JUnitArchiver however it won't record what >>> stage it captured them from (there is an outstanding Improvement in JIRA >>> for this). >>> >>> I hope I didn't miss anything >>> >>> On Thu, Aug 18, 2016 at 2:19 PM Joshua Harlow <[email protected]> wrote: >>> >>>> I to would like something like this, >>>> >>>> It seems to be a common thing to do and a real weakness of the pipeline >>>> style, >>>> >>>> If people are supposed to setup complex pipelines in a pipeline file >>>> then its pretty important that there be a way to ship out multiple result >>>> files, >>>> >>>> Even if they are successful, discarding information, especially in >>>> jenkins seems bad. >>>> >>>> Because in my view the 'Build-->Test-->Integration Test-->Deploy' is a >>>> real simple example and I personally plan on doing a much bigger pipeline >>>> like, >>>> >>>> Build project Y--> Unit Test Y--> Local functional test Y >>>> >>>> | >>>> >>>> ---> Integration Test Y + Z -> Perf test Y + Z -> Request deploy to >>>> stage ---> (wait) -> Integration Test -> Perf test --> Deploy to prod ---> >>>> (wait) --> .... >>>> >>>> | >>>> Build project Z-->Unit Test Z--> Local functional test Z >>>> >>>> And in that there will be more than Y and Z (all done in parallel with >>>> a join), at each stage having test results, >>>> >>>> Now maybe pipelines aren't meant to approach this complexity? Though if >>>> they are not, then I'm not quite sure what they are really useful for >>>> (because I can do the same thing in ansible/equivalent), >>>> >>>> -Josh >>>> >>>> >>>> On Sunday, July 24, 2016 at 1:47:37 PM UTC-7, Arnaud wrote: >>>> >>>>> Yes, like Matrix job but not only. Separate tests by stage between >>>>> unit tests and integration tests looks important for me. >>>>> >>>>> Arnaud. >>>>> >>>> 2016-07-24 21:12 GMT+02:00 James Dumay <[email protected]>: >>>>> >>>> Are you describing a situation here that's closer to what the Matrix >>>>>> job type caters to? Eg Operating System X Browser Version or something? >>>>>> >>>>>> If you don't mind drawing a picture, that could help me get across >>>>>> what you're use case entails :) >>>>>> >>>>>> On Mon, 25 Jul 2016 at 4:24 AM, Arnaud bourree <[email protected]> >>>>>> wrote: >>>>>> >>>>> Hi >>>>>>> >>>>>>> I'm interested in test grouped by stage: we'd like to run the same >>>>>>> set of test in different context: tests have same name and each can >>>>>>> work in >>>>>>> one context and not in other. Grouped by stage we can see easily which >>>>>>> context has issue >>>>>>> >>>>>>> Regards >>>>>>> >>>>>>> Arnaud Bourrée >>>>>>> >>>>>>> Le 22 juil. 2016 14:07, "ST" <[email protected]> a écrit : >>>>>>> >>>>>> James, >>>>>>> >>>>>>> Interesting statement "successful tests is an edge case" - I've >>>>>>> never thought about it this way, but if all tests pass I dont need to >>>>>>> know >>>>>>> anything else, so true I guess :-) >>>>>>> >>>>>>> >>>>>>> Our use case is (with daily releases): >>>>>>> * unit tests >>>>>>> * itests that set up their own data (most of our itests) >>>>>>> * itests relying on specific test data (i.e. they have to be run >>>>>>> against a specific test database) >>>>>>> * itests that test typical API consumption (to ensure we haven't >>>>>>> broken existing API) >>>>>>> >>>>>>> So when: >>>>>>> (A) a test fails I'd like to know into which category it falls to >>>>>>> better assess any consequences and being able to make a first guess >>>>>>> about >>>>>>> what the root cause could be >>>>>>> (B) many tests fail it's useful to know whether all of them fall >>>>>>> into same test category. >>>>>>> >>>>>>> Putting tests into a specific java package (*.itest.api.* etc.) >>>>>>> would probably help to solve (A) but not (B). >>>>>>> >>>>>>> Best regards, >>>>>>> stefan. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Jul 21, 2016 at 7:12 PM, James Dumay <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Hi Stefan, >>>>>>>> >>>>>>>> There is no way to do this with Pipeline today. However, when >>>>>>>> designing the new test reporting screen >>>>>>>> <https://jenkins.io/blog/2016/07/19/blue-ocean-update/> in Blue >>>>>>>> Ocean I did have someone ask if we could group them by stage. >>>>>>>> >>>>>>>> The problem I saw here is that most peoples pipelines look like >>>>>>>> this: >>>>>>>> >>>>>>>> Build-->Test-->Integration Test-->Deploy >>>>>>>> >>>>>>>> If any tests fail in "Test" then you would not see any tests >>>>>>>> displayed for "Integration Tests" as they were not executed - >>>>>>>> rendering the >>>>>>>> grouping reasonably useless except for viewing successful tests. >>>>>>>> (Viewing successful >>>>>>>> tests is an edge case >>>>>>>> <https://issues.jenkins-ci.org/browse/JENKINS-36299> we've opted >>>>>>>> not to handle for the moment due to UI scaling concerns). >>>>>>>> >>>>>>>> Whats your use case for grouping tests by stage? >>>>>>>> >>>>>>>> >>>>>>>> On Thursday, July 21, 2016 at 7:24:39 AM UTC-7, ST wrote: >>>>>>>>> >>>>>>>>> Hi! >>>>>>>>> >>>>>>>>> I am migrating our java/maven build pipeline to a Jenkinsfile, so >>>>>>>>> both the unit tests (maven-surefire-plugin) and integration tests >>>>>>>>> (maven-failsafe-plugin) will be run inside the same pipeline, but in >>>>>>>>> different stages. >>>>>>>>> >>>>>>>>> How can I separate the test results for unit tests and integration >>>>>>>>> tests in Jenkins? I am collecting test results like this: >>>>>>>>> >>>>>>>>> for unit tests: >>>>>>>>> step([$class: 'JUnitResultArchiver', testResults: >>>>>>>>> '**/target/surefire-reports/*.xml']) >>>>>>>>> >>>>>>>>> for integration tests: >>>>>>>>> step([$class: 'JUnitResultArchiver', testResults: >>>>>>>>> '**/target/failsafe-reports/*.xml']) >>>>>>>>> >>>>>>>>> And they end up as a mixed list under "Test Result" for every >>>>>>>>> build. >>>>>>>>> >>>>>>>>> Since they were separate jobs before, they were separated in >>>>>>>>> Jenkins as well. Would be nice to have them as separated list also in >>>>>>>>> the >>>>>>>>> Jenkins 2.x pipeline, but how? >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> stefan. >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Jenkins Users" group. >>>>>>>> >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>> >>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/c48d03ee-fe43-4538-a875-46406f65f601%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/c48d03ee-fe43-4538-a875-46406f65f601%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Jenkins Users" group. >>>>>>> >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/CABwQARv-Fnh7j4YucDnzWc-uDC16wekW29E4E5FD9o6kPSsGtQ%40mail.gmail.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/CABwQARv-Fnh7j4YucDnzWc-uDC16wekW29E4E5FD9o6kPSsGtQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>>> -- >>>>>>> >>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "Jenkins Users" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/jenkinsci-users/CgTuhpm5sZY/unsubscribe >>>>>>> . >>>>>>> >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> [email protected]. >>>>>> >>>>>> >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/CAESUf_BH5Yvpx-VoMWeHe%3DpT%2BiorKfqucjZty%2BOeYdiOhgtvmg%40mail.gmail.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/CAESUf_BH5Yvpx-VoMWeHe%3DpT%2BiorKfqucjZty%2BOeYdiOhgtvmg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>>> James Dumay >>>>>> Product Manager >>>>>> CloudBees, Inc. >>>>>> Skype: jdumaycb Twitter: i386 >>>>>> >>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Jenkins Users" group. >>>>>> >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to [email protected]. >>>>>> >>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/jenkinsci-users/CAErbOvSxgVrj_DqNCVxyBoHJV-WVszkM8DmeDbjUv%3DXfW9b7DQ%40mail.gmail.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/CAErbOvSxgVrj_DqNCVxyBoHJV-WVszkM8DmeDbjUv%3DXfW9b7DQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Jenkins Users" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/jenkinsci-users/CgTuhpm5sZY/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/jenkinsci-users/8b2d8171-2f75-44d2-908d-f701fa131744%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/jenkinsci-users/8b2d8171-2f75-44d2-908d-f701fa131744%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> James Dumay >>> Product Manager >>> CloudBees, Inc. >>> Skype: jdumaycb Twitter: i386 >>> >> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7e73c195-b727-419e-854a-a53df4256cf2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
