Yes, like Matrix job but not only. Separate tests by stage between unit tests and integration tests looks important for me.
Arnaud. 2016-07-24 21:12 GMT+02:00 James Dumay <[email protected]>: > Are you describing a situation here that's closer to what the Matrix job > type caters to? Eg Operating System X Browser Version or something? > > If you don't mind drawing a picture, that could help me get across what > you're use case entails :) > > On Mon, 25 Jul 2016 at 4:24 AM, Arnaud bourree <[email protected]> > wrote: > >> Hi >> >> I'm interested in test grouped by stage: we'd like to run the same set of >> test in different context: tests have same name and each can work in one >> context and not in other. Grouped by stage we can see easily which context >> has issue >> >> Regards >> >> Arnaud Bourrée >> >> Le 22 juil. 2016 14:07, "ST" <[email protected]> a écrit : >> >> James, >> >> Interesting statement "successful tests is an edge case" - I've never >> thought about it this way, but if all tests pass I dont need to know >> anything else, so true I guess :-) >> >> >> Our use case is (with daily releases): >> * unit tests >> * itests that set up their own data (most of our itests) >> * itests relying on specific test data (i.e. they have to be run against >> a specific test database) >> * itests that test typical API consumption (to ensure we haven't broken >> existing API) >> >> So when: >> (A) a test fails I'd like to know into which category it falls to better >> assess any consequences and being able to make a first guess about what the >> root cause could be >> (B) many tests fail it's useful to know whether all of them fall into >> same test category. >> >> Putting tests into a specific java package (*.itest.api.* etc.) would >> probably help to solve (A) but not (B). >> >> Best regards, >> stefan. >> >> >> >> >> On Thu, Jul 21, 2016 at 7:12 PM, James Dumay <[email protected]> >> wrote: >> >>> Hi Stefan, >>> >>> There is no way to do this with Pipeline today. However, when designing >>> the new test reporting screen >>> <https://jenkins.io/blog/2016/07/19/blue-ocean-update/> in Blue Ocean I >>> did have someone ask if we could group them by stage. >>> >>> The problem I saw here is that most peoples pipelines look like this: >>> >>> Build-->Test-->Integration Test-->Deploy >>> >>> If any tests fail in "Test" then you would not see any tests displayed >>> for "Integration Tests" as they were not executed - rendering the grouping >>> reasonably useless except for viewing successful tests. (Viewing successful >>> tests is an edge case >>> <https://issues.jenkins-ci.org/browse/JENKINS-36299> we've opted not to >>> handle for the moment due to UI scaling concerns). >>> >>> Whats your use case for grouping tests by stage? >>> >>> >>> On Thursday, July 21, 2016 at 7:24:39 AM UTC-7, ST wrote: >>>> >>>> Hi! >>>> >>>> I am migrating our java/maven build pipeline to a Jenkinsfile, so both >>>> the unit tests (maven-surefire-plugin) and integration tests >>>> (maven-failsafe-plugin) will be run inside the same pipeline, but in >>>> different stages. >>>> >>>> How can I separate the test results for unit tests and integration >>>> tests in Jenkins? I am collecting test results like this: >>>> >>>> for unit tests: >>>> step([$class: 'JUnitResultArchiver', testResults: >>>> '**/target/surefire-reports/*.xml']) >>>> >>>> for integration tests: >>>> step([$class: 'JUnitResultArchiver', testResults: >>>> '**/target/failsafe-reports/*.xml']) >>>> >>>> And they end up as a mixed list under "Test Result" for every build. >>>> >>>> Since they were separate jobs before, they were separated in Jenkins as >>>> well. Would be nice to have them as separated list also in the Jenkins 2.x >>>> pipeline, but how? >>>> >>>> Best regards, >>>> stefan. >>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/c48d03ee-fe43-4538-a875-46406f65f601%40googlegroups.com >>> <https://groups.google.com/d/msgid/jenkinsci-users/c48d03ee-fe43-4538-a875-46406f65f601%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/jenkinsci-users/CABwQARv-Fnh7j4YucDnzWc-uDC16wekW29E4E5FD9o6kPSsGtQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/jenkinsci-users/CABwQARv-Fnh7j4YucDnzWc-uDC16wekW29E4E5FD9o6kPSsGtQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Jenkins Users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/jenkinsci-users/CgTuhpm5sZY/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/jenkinsci-users/CAESUf_BH5Yvpx-VoMWeHe%3DpT%2BiorKfqucjZty%2BOeYdiOhgtvmg%40mail.gmail.com >> <https://groups.google.com/d/msgid/jenkinsci-users/CAESUf_BH5Yvpx-VoMWeHe%3DpT%2BiorKfqucjZty%2BOeYdiOhgtvmg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > James Dumay > Product Manager > CloudBees, Inc. > Skype: jdumaycb Twitter: i386 > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-users/CAErbOvSxgVrj_DqNCVxyBoHJV-WVszkM8DmeDbjUv%3DXfW9b7DQ%40mail.gmail.com > <https://groups.google.com/d/msgid/jenkinsci-users/CAErbOvSxgVrj_DqNCVxyBoHJV-WVszkM8DmeDbjUv%3DXfW9b7DQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAESUf_Dn1Oz0ao4FL-egkx0-bD%3DtZLS3P%3DAQBvE3nT8pB-rRMA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
