It also gets interesting that if I have 2 runs (pre-patch unit testing and post-patch unittest) but both have tests with the same name in them,
Unsure exactly how that will render :-P I'll see what I can find out, other option I guess is to just publish a bunch of html (and not use the results renderer). -Josh On Wednesday, August 17, 2016 at 10:19:13 PM UTC-7, Joshua Harlow wrote: > > Do I need to just give each capture a unique filename? > > I was trying this earlier and it appears giving them the same name only > caused the first to show up on the UI :( > > Perhaps though I was doing something wrong... > > On Wednesday, August 17, 2016 at 9:24:37 PM UTC-7, James Dumay wrote: >> >> Joshua, I don't think there is anything stopping you from recording the >> test results in the Pipeline you have described. Jenkins will capture them >> wherever you ask it to with the JUnitArchiver however it won't record what >> stage it captured them from (there is an outstanding Improvement in JIRA >> for this). >> >> I hope I didn't miss anything >> >> On Thu, Aug 18, 2016 at 2:19 PM Joshua Harlow <[email protected]> wrote: >> >>> I to would like something like this, >>> >>> It seems to be a common thing to do and a real weakness of the pipeline >>> style, >>> >>> If people are supposed to setup complex pipelines in a pipeline file >>> then its pretty important that there be a way to ship out multiple result >>> files, >>> >>> Even if they are successful, discarding information, especially in >>> jenkins seems bad. >>> >>> Because in my view the 'Build-->Test-->Integration Test-->Deploy' is a >>> real simple example and I personally plan on doing a much bigger pipeline >>> like, >>> >>> Build project Y--> Unit Test Y--> Local functional test Y >>> >>> | >>> >>> ---> Integration Test Y + Z -> Perf test Y + Z -> Request deploy to >>> stage ---> (wait) -> Integration Test -> Perf test --> Deploy to prod ---> >>> (wait) --> .... >>> >>> | >>> Build project Z-->Unit Test Z--> Local functional test Z >>> >>> And in that there will be more than Y and Z (all done in parallel with a >>> join), at each stage having test results, >>> >>> Now maybe pipelines aren't meant to approach this complexity? Though if >>> they are not, then I'm not quite sure what they are really useful for >>> (because I can do the same thing in ansible/equivalent), >>> >>> -Josh >>> >>> >>> On Sunday, July 24, 2016 at 1:47:37 PM UTC-7, Arnaud wrote: >>> >>>> Yes, like Matrix job but not only. Separate tests by stage between unit >>>> tests and integration tests looks important for me. >>>> >>>> Arnaud. >>>> >>> 2016-07-24 21:12 GMT+02:00 James Dumay <[email protected]>: >>>> >>> Are you describing a situation here that's closer to what the Matrix job >>>>> type caters to? Eg Operating System X Browser Version or something? >>>>> >>>>> If you don't mind drawing a picture, that could help me get across >>>>> what you're use case entails :) >>>>> >>>>> On Mon, 25 Jul 2016 at 4:24 AM, Arnaud bourree <[email protected]> >>>>> wrote: >>>>> >>>> Hi >>>>>> >>>>>> I'm interested in test grouped by stage: we'd like to run the same >>>>>> set of test in different context: tests have same name and each can work >>>>>> in >>>>>> one context and not in other. Grouped by stage we can see easily which >>>>>> context has issue >>>>>> >>>>>> Regards >>>>>> >>>>>> Arnaud Bourrée >>>>>> >>>>>> Le 22 juil. 2016 14:07, "ST" <[email protected]> a écrit : >>>>>> >>>>> James, >>>>>> >>>>>> Interesting statement "successful tests is an edge case" - I've never >>>>>> thought about it this way, but if all tests pass I dont need to know >>>>>> anything else, so true I guess :-) >>>>>> >>>>>> >>>>>> Our use case is (with daily releases): >>>>>> * unit tests >>>>>> * itests that set up their own data (most of our itests) >>>>>> * itests relying on specific test data (i.e. they have to be run >>>>>> against a specific test database) >>>>>> * itests that test typical API consumption (to ensure we haven't >>>>>> broken existing API) >>>>>> >>>>>> So when: >>>>>> (A) a test fails I'd like to know into which category it falls to >>>>>> better assess any consequences and being able to make a first guess >>>>>> about >>>>>> what the root cause could be >>>>>> (B) many tests fail it's useful to know whether all of them fall into >>>>>> same test category. >>>>>> >>>>>> Putting tests into a specific java package (*.itest.api.* etc.) would >>>>>> probably help to solve (A) but not (B). >>>>>> >>>>>> Best regards, >>>>>> stefan. >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jul 21, 2016 at 7:12 PM, James Dumay <[email protected]> >>>>>> wrote: >>>>>> >>>>>> Hi Stefan, >>>>>>> >>>>>>> There is no way to do this with Pipeline today. However, when >>>>>>> designing the new test reporting screen >>>>>>> <https://jenkins.io/blog/2016/07/19/blue-ocean-update/> in Blue >>>>>>> Ocean I did have someone ask if we could group them by stage. >>>>>>> >>>>>>> The problem I saw here is that most peoples pipelines look like this: >>>>>>> >>>>>>> Build-->Test-->Integration Test-->Deploy >>>>>>> >>>>>>> If any tests fail in "Test" then you would not see any tests >>>>>>> displayed for "Integration Tests" as they were not executed - rendering >>>>>>> the >>>>>>> grouping reasonably useless except for viewing successful tests. >>>>>>> (Viewing successful >>>>>>> tests is an edge case >>>>>>> <https://issues.jenkins-ci.org/browse/JENKINS-36299> we've opted >>>>>>> not to handle for the moment due to UI scaling concerns). >>>>>>> >>>>>>> Whats your use case for grouping tests by stage? >>>>>>> >>>>>>> >>>>>>> On Thursday, July 21, 2016 at 7:24:39 AM UTC-7, ST wrote: >>>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> I am migrating our java/maven build pipeline to a Jenkinsfile, so >>>>>>>> both the unit tests (maven-surefire-plugin) and integration tests >>>>>>>> (maven-failsafe-plugin) will be run inside the same pipeline, but in >>>>>>>> different stages. >>>>>>>> >>>>>>>> How can I separate the test results for unit tests and integration >>>>>>>> tests in Jenkins? I am collecting test results like this: >>>>>>>> >>>>>>>> for unit tests: >>>>>>>> step([$class: 'JUnitResultArchiver', testResults: >>>>>>>> '**/target/surefire-reports/*.xml']) >>>>>>>> >>>>>>>> for integration tests: >>>>>>>> step([$class: 'JUnitResultArchiver', testResults: >>>>>>>> '**/target/failsafe-reports/*.xml']) >>>>>>>> >>>>>>>> And they end up as a mixed list under "Test Result" for every build. >>>>>>>> >>>>>>>> Since they were separate jobs before, they were separated in >>>>>>>> Jenkins as well. Would be nice to have them as separated list also in >>>>>>>> the >>>>>>>> Jenkins 2.x pipeline, but how? >>>>>>>> >>>>>>>> Best regards, >>>>>>>> stefan. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Jenkins Users" group. >>>>>>> >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>> >>>>>> >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/c48d03ee-fe43-4538-a875-46406f65f601%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/c48d03ee-fe43-4538-a875-46406f65f601%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Jenkins Users" group. >>>>>> >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/jenkinsci-users/CABwQARv-Fnh7j4YucDnzWc-uDC16wekW29E4E5FD9o6kPSsGtQ%40mail.gmail.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/CABwQARv-Fnh7j4YucDnzWc-uDC16wekW29E4E5FD9o6kPSsGtQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> -- >>>>>> >>>>> You received this message because you are subscribed to a topic in the >>>>>> Google Groups "Jenkins Users" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/jenkinsci-users/CgTuhpm5sZY/unsubscribe >>>>>> . >>>>>> >>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> [email protected]. >>>>> >>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/jenkinsci-users/CAESUf_BH5Yvpx-VoMWeHe%3DpT%2BiorKfqucjZty%2BOeYdiOhgtvmg%40mail.gmail.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/CAESUf_BH5Yvpx-VoMWeHe%3DpT%2BiorKfqucjZty%2BOeYdiOhgtvmg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>>> James Dumay >>>>> Product Manager >>>>> CloudBees, Inc. >>>>> Skype: jdumaycb Twitter: i386 >>>>> >>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Jenkins Users" group. >>>>> >>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/jenkinsci-users/CAErbOvSxgVrj_DqNCVxyBoHJV-WVszkM8DmeDbjUv%3DXfW9b7DQ%40mail.gmail.com >>>>> >>>>> <https://groups.google.com/d/msgid/jenkinsci-users/CAErbOvSxgVrj_DqNCVxyBoHJV-WVszkM8DmeDbjUv%3DXfW9b7DQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Jenkins Users" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/jenkinsci-users/CgTuhpm5sZY/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/8b2d8171-2f75-44d2-908d-f701fa131744%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/jenkinsci-users/8b2d8171-2f75-44d2-908d-f701fa131744%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> James Dumay >> Product Manager >> CloudBees, Inc. >> Skype: jdumaycb Twitter: i386 >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b2a9715c-661b-49d0-8e6b-f5d80b69eeeb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
