On Sat, 7 Jan 2023 at 10:44 AM, 'Alexey Savchkov' via go-cd <
[email protected]> wrote:
> Thank you for the detailed response.
>
> > a test failure means the report would either be incomplete or not useful
> isn't it?
> No, by a test failure I mean the negative result of a test ("Test passed"
> / "Test failed"). From the programmatical point of view there is no failure
> so the resulting test reports as such are intact and just show that some
> tests have failed (not passed).
>
> As far as I can see adding a dummy task with "runIf = any" doesn't prevent
> the stage from going into the failed state and terminating the pipeline.
> The intended next stage is not started, see below.
>
> [image: post-stage-run.png]
> [image: post-stage-log.png]
> A manual trigger for generating reports is not an option for me, I'm
> affraid. The reports are an essential part of our daily work and are
> expected to be available without manually processing them for every build.
>
> I still think that the only production-level solution to avoid such
> workarounds would be to have optional "post" jobs for each stage and for
> the entire pipeline. These jobs would be then run after the completeion of
> all previous taks (jobs in a stage or stages in a pipeline) and
> additionally controlled by a similar "run If" attribute for flexibility.
> One side not with respect to this is that now when a build is cancelled
> then even "runIf = any" jobs are not run. A "post" job with "runIf = any"
> should run even if a build is cancelled as there may be cleanup steps in it
> which must be always performed.
> Also found this
> <https://groups.google.com/g/go-cd/c/V6-LfGe1yjM/m/LygoP3VHAQAJ>discussion
> showing that something like this is in demand.
>
Until such a feature is implemented, you could consider the following:
- capture the exit code and save as an artifact
- exit with zero ( to indicate a successful run )
- have a job in the next stage gather all the test reports from the earlier
stage and merge them
- have another job fetch all the exit code artifacts from the jobs in the
earlier stages and then exit 1 if any of those exit codes were zero.
This way,
- all the tests run
- the test artifacts are available in the next stage
- the overall pipeline fails while making test execution reports available
as an aggregate.
> суббота, 7 января 2023 г. в 06:23:25 UTC+7, [email protected]:
>
>> Under ideal conditions, I would assume a test failure means the report
>> would either be incomplete or not useful isn't it?
>>
>> AFAIK there isn't a post-build job for a stage, you can do it at a job
>> level though using "Run If" Attribute of the tasks. Here are a few ways to
>> get what you need, it might range from ugly to "Oh, that's interesting". If
>> you do decide to choose these methods please drop a line so I know if it
>> was useful :-)
>>
>> *Option 1*:
>> Assuming you can tell from the reports if a test has failed or not, you
>> just have a dummy task after the tests that runs on runif="any" and do the
>> aggregation on the next stage. If you know from the reports something has
>> failed you can always fail the aggregate stage with some useful
>> information.
>> *Cons*: The failure will not immediately tell you what failed but again
>> if the folks who need to view the error have access to GoCD logs they can
>> always view the error and take necessary actions.
>>
>> *Option 2:*
>> Very similar to Option 1, but if you can't tell from the reports if a
>> test has failed then you can emit an artifact that says if a test has
>> failed or not and then do the same in the aggregate stage to parse these
>> artifacts and then decide to fail the stage.
>> *Cons*: If the previous option wasn't ugly enough, this definitely feels
>> that way for me.
>>
>> *Option 3:*
>> Have a separate stage after the tests that does the aggregation, instead
>> of doing all the gymnastics with artifacts or run if attributes, you can
>> write a notification plugin that listens to *only failure* events of
>> this stage and then manually trigger the next stage.
>> *Cons*: Effort to writing to a plugin and maintaining it. Let me know if
>> you are comfortable contributing this to the gocd-slack-notifier
>> <https://github.com/ashwanthkumar/gocd-slack-build-notifier> plugin.
>> Happy to collaborate on this if required too.
>>
>> Thanks,
>>
>>
>> On Fri, 6 Jan 2023 at 18:54, 'Alexey Savchkov' via go-cd <
>> [email protected]> wrote:
>>
>>> I'm stuck with a problem which I can't think of a good solution for so
>>> would like to know your thoughts on the subject.
>>> One stage of my pipeline is a test matrix consisting of 6 jobs. Each job
>>> produces an Allure test report. Now I need to collect these individual test
>>> reports and build an aggregate one from them. Normally I would do it in
>>> some kind of a post-build job running at the end of each stage or the
>>> entire pipeline. But as far as I see there are no such post-build jobs in
>>> GoCD. I also can't use a separate stage for this since it won't start if
>>> the test stage fails due to failed tests. The only possibility I see is to
>>> start a dedicated job (say, "Report") in the test stage together with the
>>> test jobs, make it continuously poll the states of the test jobs and when
>>> all of them are completed fetch their report artifacts and build the total
>>> report. Wasting of runners when they are just waiting for the test jobs to
>>> finish but still acceptable.
>>> Am I not missing anything very basic as to how to go about such cases in
>>> GoCD? What would you propose? An external storage (Nexus) is available if
>>> this matters.
>>>
>>> Thank you very much in advance.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/d70ab788-59d1-45d3-a088-77b97b3fe309n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/d70ab788-59d1-45d3-a088-77b97b3fe309n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
>>
>> --
>>
>> Ashwanth Kumar / ashwanthkumar.in
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/06bccc69-242b-424b-8769-c4c9f46caedan%40googlegroups.com
> <https://groups.google.com/d/msgid/go-cd/06bccc69-242b-424b-8769-c4c9f46caedan%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/go-cd/CANiY96ax1R-Qg2xY%3Drmow-5BCox%3DuCxJr37pBUrhmNaVD6jGBw%40mail.gmail.com.
