Just thinking out loud, what if the integration test pipeline was refactored to have two stages. First stage to run the integration-tests and second to publish the changed installers to a package repository ( https://www.go.cd/plugins/). 'DeploymentService1' pipeline could now be changed to depend on this package repo instead with appropriate package spec (say Service1) specified. The same goes for the other two deployment pipelines - DeploymentService2 and DeploymentService3. With this, the deployment pipelines would get triggered only if a new package matching the specified specs was published. Won't that help simply the scenario?
On Wednesday, 5 October 2016 15:59:54 UTC+5:30, Anant Agarwal wrote: > > May be https://api.go.cd/current/#scheduling-pipelines is the answer > for controlling the deployment pipelines trigger. > > > On Wednesday, October 5, 2016 at 9:50:05 AM UTC+1, Anant Agarwal wrote: >> >> Hi Aravind >> >> Thank you very for your response. I like the idea you have suggested but >> unable to comment on the code snippet you have attached. >> Could you clarify following in bold "as an artefact to all three >> downstream pipelines, *which can figure out whether to re-deploy or not.*" >> does the pipeline has ability to figure out to run or not based on >> artefact or are you suggesting that deployment code can decide whether to >> deploy or not in which case all deployment pipeline will run but deployment >> code will deploy conditionally. >> >> I was thinking that may be I have to rethink how I setup the pipeline, I >> am finding it hard to believe that such a common scenario in today's word >> about building microservices, running integration test and deployment of >> what has changed should be easy. >> >> Thanks >> Anant Agarwal >> >> >> On Monday, October 3, 2016 at 9:51:08 PM UTC+1, Aravind SV wrote: >>> >>> No, there's no way, as far as I know. I remember that this question (or >>> a very similar one) has come up before. I think what's making this hard to >>> do is the fact that the information shown in the build cause popup of the >>> "IntegrationTest" pipeline is lost. I mean, if you click on "Changes" in >>> that pipeline, it'll show you, in yellow, which upstream pipeline(s) caused >>> it ("IntegrationTest" pipeline) to trigger. But, once that pipeline has >>> finished, that information is no longer accessible to the downstream >>> pipelines (DeploymentService1, ...). >>> >>> I'm wondering: If GoCD provided an environment variable (say, >>> "GO_material_name_CHANGED" like "GO_material_name_LABEL" etc), then a >>> deployment descriptor (a file describing what happened) can be created and >>> pushed down as an artifact to all three downstream pipelines, which can >>> figure out whether to re-deploy or not. >>> >>> I know it's not a pipeline modeling solution but it is a solution which >>> can enable this kind of a workflow to be supported. Thoughts? >>> >>> >>> I'm thinking of a code change like this (just to clarify): >>> >>> diff --git >>> a/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java >>> b/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java >>> index 2513d68..3847bf2 100644--- >>> a/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java+++ >>> b/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java@@ >>> -168,6 +168,7 @@ public class PackageMaterial extends AbstractMaterial { >>> @Override >>> public void populateEnvironmentContext(EnvironmentVariableContext >>> context, MaterialRevision materialRevision, File workingDir) { >>> context.setProperty(upperCase(format("GO_PACKAGE_%s_LABEL", >>> escapeEnvironmentVariable(getName().toString()))), >>> materialRevision.getRevision().getRevision(), false);+ >>> context.setProperty(upperCase(format("GO_PACKAGE_%s_CHANGED", >>> escapeEnvironmentVariable(getName().toString()))), >>> Boolean.toString(materialRevision.isChanged()), false); >>> for (ConfigurationProperty configurationProperty : >>> getPackageDefinition().getRepository().getConfiguration()) { >>> context.setProperty(getEnvironmentVariableKey("GO_REPO_%s_%s", >>> configurationProperty.getConfigurationKey().getName()), >>> configurationProperty.getValue(), >>> configurationProperty.isSecure()); >>> diff --git >>> a/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java >>> b/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java >>> index 391bfae..1190cf9 100644--- >>> a/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java+++ >>> >>> b/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java@@ >>> -265,6 +265,7 @@ public class PluggableSCMMaterial extends >>> AbstractMaterial { >>> @Override >>> public void populateEnvironmentContext(EnvironmentVariableContext >>> context, MaterialRevision materialRevision, File workingDir) { >>> context.setProperty(getEnvironmentVariableKey("GO_SCM_%s_%s", >>> "LABEL"), materialRevision.getRevision().getRevision(), false);+ >>> context.setProperty(getEnvironmentVariableKey("GO_SCM_%s_%s", "CHANGED"), >>> Boolean.toString(materialRevision.isChanged()), false); >>> for (ConfigurationProperty configurationProperty : >>> scmConfig.getConfiguration()) { >>> context.setProperty(getEnvironmentVariableKey("GO_SCM_%s_%s", >>> configurationProperty.getConfigurationKey().getName()), >>> configurationProperty.getValue(), >>> configurationProperty.isSecure()); >>> diff --git >>> a/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java >>> b/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java >>> index d3c86db..aa9509e 100644--- >>> a/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java+++ >>> b/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java@@ >>> -129,13 +129,14 @@ public abstract class ScmMaterial extends >>> AbstractMaterial { >>> String toRevision = materialRevision.getRevision().getRevision(); >>> String fromRevision = >>> materialRevision.getOldestRevision().getRevision(); >>> - setGoRevisionVariables(environmentVariableContext, fromRevision, >>> toRevision);+ setGoRevisionVariables(environmentVariableContext, >>> materialRevision, fromRevision, toRevision); >>> } >>> - private void setGoRevisionVariables(EnvironmentVariableContext >>> environmentVariableContext, String fromRevision, String toRevision) {+ >>> private void setGoRevisionVariables(EnvironmentVariableContext >>> environmentVariableContext, MaterialRevision materialRevision, String >>> fromRevision, String toRevision) { >>> setVariableWithName(environmentVariableContext, toRevision, >>> GO_REVISION); >>> setVariableWithName(environmentVariableContext, toRevision, >>> GO_TO_REVISION); >>> setVariableWithName(environmentVariableContext, fromRevision, >>> GO_FROM_REVISION);+ setVariableWithName(environmentVariableContext, >>> Boolean.toString(materialRevision.isChanged()), "GO_CHANGED"); >>> } >>> >>> protected void setVariableWithName(EnvironmentVariableContext >>> environmentVariableContext, String value, String propertyName) { >>> diff --git >>> a/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java >>> >>> b/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java >>> index 016e049..ea3681a 100644--- >>> a/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java+++ >>> >>> b/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java@@ >>> -160,6 +160,7 @@ public class DependencyMaterial extends AbstractMaterial { >>> DependencyMaterialRevision revision = (DependencyMaterialRevision) >>> materialRevision.getRevision(); >>> context.setPropertyWithEscape(format("GO_DEPENDENCY_LABEL_%s", >>> getName()), revision.getPipelineLabel()); >>> context.setPropertyWithEscape(format("GO_DEPENDENCY_LOCATOR_%s", >>> getName()), revision.getRevision());+ >>> context.setPropertyWithEscape(format("GO_DEPENDENCY_CHANGED_%s", >>> getName()), Boolean.toString(materialRevision.isChanged())); >>> } >>> >>> public boolean isAutoUpdate() { >>> >>> >>> >>> >>> >>> On Mon, Oct 3, 2016 at 6:01 AM, 'Anant Agarwal' via go-cd < >>> [email protected]> wrote: >>> >>>> Hi >>>> >>>> We have a bunch of micro services in separate code repositories. The >>>> pipelines we have setup look like the diagram below. >>>> >>>> <https://lh3.googleusercontent.com/-n18NACBsIBY/V_IrFsQqmCI/AAAAAAAAAmU/IwkT90HzjAcdeJXPz22XnMJlHtxJXd6TQCLcB/s1600/Screen%2BShot%2B2016-10-03%2Bat%2B10.51.46.png> >>>> >>>> At the moment whenever we make changes to any one service let's say >>>> service1, integration test pipeline runs and then triggers deployment of >>>> all services. >>>> >>>> What we want is that Integration test pipeline should run if changes to >>>> any of the dependent services but it should trigger deployment only for >>>> the >>>> particular service which originated the change. Is that possible? >>>> >>>> so we are kind of looking to achieve "AND" condition in pipeline >>>> trigger. i.e. Deployment service 1 pipeline should trigger only when >>>> Service1 Artefact was built AND integration test passed. and this should >>>> not trigger Deployment service2 pipeline. >>>> >>>> >>>> Appreciate your help. >>>> >>>> >>>> Thanks >>>> Anant Agarwal >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
