You could upload it from every stage that passes, but I don't see a benefit to doing that -- and as you said it's duplicative. What I'm suggesting is that every downstream stage/pipeline can use the build stage as a material so you can fetch artifacts from it.
Something like: build-and-test (generates artifact) --> deploy QA --> validate QA --> promote-to-prod-manual-gate --> deploy-prod Every stage that needs to use the built artifact can include build-and-test as a material so that you can do a Fetch Artifact task. I believe you had a concern/question that it meant stages downstream would have access to an artifact that possibly didn't pass validations? I don't think that's an issue because you would have more than 1 material. there's an implicit dependency on the prior stage succeeding by default. For instance, "deploy-prod" would only run if all the prior stages succeeded, I believe, so it wouldn't actually deploy without going through all of your validation stages (deploy-QA, validate-QA, promote-to-prod-manual-gate). So in other words, deploy-prod would need to depend on both promote-to-prod-manual-gate and build-and-test. If you can't get it working, let me dust off my GoCD hat and look at it tonight after working hours. Best, Marques On Wed, Dec 14, 2022 at 10:26 AM Anh Le <[email protected]> wrote: > 1. So you are suggesting there is no way to avoid temporarily duplicating > the artifact between pipelines following my use case right? I'm still new > to GoCD and haven't play much with plugins > > Not sure if I get your idea correctly, does that mean if I model my > pipelines like this: > build pipeline - deploy to QA pipeline - mark passed QA Gate pipeline - > deploy to Production pipeline > If I have a perfect artifact, I will need to Publishing artifacts and *Fetch > Artifacts* for *each pipeline* > > Since storage is cheap, I'm perfectly fine with this approach. But having > to upload the same file every time could sound not optimal > > 2. Is it possible to avoid that duplication? (but it could be > over-engineer, I'm not sure, haven't spent much time doing this after > reading the Continuous Delivery book) > > I'm thinking is there any vendor/tool/plugins that does artifact > management strategy and could be integrated well with GoCD OOTB: > - An artifact (it is a file generated from the build step anyway) should > have some states: > + Deployed to QA > + Passed QA Gate > Mark passed QA Gate pipeline and Deploy to Production > pipeline should fetch artifacts based on that state from a central > Artifactory > On Wednesday, 14 December 2022 at 22:54:46 UTC+7 [email protected] > wrote: > >> It’s been a few years since I really looked at GoCD, so my memory is hazy >> on this. >> >> You should be able to store artifacts on the GoCD server itself by >> configuring a build artifact path. Then downstream stages or pipelines can >> grab them from gocd server directly. There’s an S3 plugin as well if you >> wanted to persist them in AWS. Then you can upload to artifactory when >> you’ve had an artifact pass certain stages for confidence. >> >> You should be able to pass artifacts through stages so that a downstream >> stage can fetch an artifact that passed the previous stage. If a stage >> depends on the previous stage passing, it cannot run until those conditions >> are met, so they won’t run with artifacts that didn’t succeed in upstream >> stages. >> >> Once it’s passed production, you can then publish it to artifactory such >> that artifactory only has production quality packages. >> >> Does that help? >> >> -Marques >> >> On Wed, Dec 14, 2022 at 7:33 AM Anh Le <[email protected]> wrote: >> >>> Hello everyone, >>> >>> I'm learning GoCD, I only know how to use the Files and Directories out >>> of the box as Artifactory. >>> I have few question about artifacts management best practices or your >>> preferred approach >>> Assumption: >>> - Already has a build pipeline (to generate artifact) >>> - Deploy to QA is manual deploy and depend on build pipeline success >>> - Deploy to Prod is manual deploy and depend on Deploy to QA pipeline >>> success >>> I'm thinking about things below but something doesn't feel right: >>> - In the build pipeline, after run build tasks, I push the artifact to >>> an Artifactory >>> + How to fetch artifact for "Deploy to QA" and "Deploy to >>> Production"? if I add build as material (so I can get the artifact) somehow >>> I feel it is weird because Production can access to something not already >>> in QA. If I publish artifact again in QA to break the dependency of Prod to >>> build then I feel that the artifact is duplicated >>> >>> What are your suggestions? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "go-cd" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/go-cd/d7155810-ffd1-4ecd-822a-0bddfb7ae419n%40googlegroups.com >>> <https://groups.google.com/d/msgid/go-cd/d7155810-ffd1-4ecd-822a-0bddfb7ae419n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/042f88ea-2e9e-4433-a2a8-af77cbbb294cn%40googlegroups.com > <https://groups.google.com/d/msgid/go-cd/042f88ea-2e9e-4433-a2a8-af77cbbb294cn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CAPKX9jZEVi2-eutVisHTF_KE353W-ZBGVvBDW%3D5LL_X0HDizgA%40mail.gmail.com.
