Thanks Jyoti so there are two key points here: 1. publish the changed installers to a package repository 2. DeploymentService1' pipeline could now be changed to depend on this package repo instead with appropriate package spec (say Service1) specified.
Your idea looks good however I am not sure about point2 "appropriate package spec" I will explore this configuration and will get back. Thanks a lot. Sent from my iPhone > On 5 Oct 2016, at 12:32, Jyoti Singh <[email protected]> wrote: > > Just thinking out loud, what if the integration test pipeline was refactored > to have two stages. First stage to run the integration-tests and second to > publish the changed installers to a package repository > (https://www.go.cd/plugins/). > 'DeploymentService1' pipeline could now be changed to depend on this package > repo instead with appropriate package spec (say Service1) specified. The same > goes for the other two deployment pipelines - DeploymentService2 and > DeploymentService3. > With this, the deployment pipelines would get triggered only if a new package > matching the specified specs was published. Won't that help simply the > scenario? > >> On Wednesday, 5 October 2016 15:59:54 UTC+5:30, Anant Agarwal wrote: >> May be https://api.go.cd/current/#scheduling-pipelines is the answer for >> controlling the deployment pipelines trigger. >> >> >>> On Wednesday, October 5, 2016 at 9:50:05 AM UTC+1, Anant Agarwal wrote: >>> Hi Aravind >>> >>> Thank you very for your response. I like the idea you have suggested but >>> unable to comment on the code snippet you have attached. >>> Could you clarify following in bold "as an artefact to all three >>> downstream pipelines, which can figure out whether to re-deploy or not." >>> does the pipeline has ability to figure out to run or not based on >>> artefact or are you suggesting that deployment code can decide whether to >>> deploy or not in which case all deployment pipeline will run but deployment >>> code will deploy conditionally. >>> >>> I was thinking that may be I have to rethink how I setup the pipeline, I >>> am finding it hard to believe that such a common scenario in today's word >>> about building microservices, running integration test and deployment of >>> what has changed should be easy. >>> >>> Thanks >>> Anant Agarwal >>> >>> >>>> On Monday, October 3, 2016 at 9:51:08 PM UTC+1, Aravind SV wrote: >>>> No, there's no way, as far as I know. I remember that this question (or a >>>> very similar one) has come up before. I think what's making this hard to >>>> do is the fact that the information shown in the build cause popup of the >>>> "IntegrationTest" pipeline is lost. I mean, if you click on "Changes" in >>>> that pipeline, it'll show you, in yellow, which upstream pipeline(s) >>>> caused it ("IntegrationTest" pipeline) to trigger. But, once that pipeline >>>> has finished, that information is no longer accessible to the downstream >>>> pipelines (DeploymentService1, ...). >>>> >>>> I'm wondering: If GoCD provided an environment variable (say, >>>> "GO_material_name_CHANGED" like "GO_material_name_LABEL" etc), then a >>>> deployment descriptor (a file describing what happened) can be created and >>>> pushed down as an artifact to all three downstream pipelines, which can >>>> figure out whether to re-deploy or not. >>>> >>>> I know it's not a pipeline modeling solution but it is a solution which >>>> can enable this kind of a workflow to be supported. Thoughts? >>>> >>>> >>>> I'm thinking of a code change like this (just to clarify): >>>> >>>> diff --git >>>> a/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java >>>> b/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java >>>> index 2513d68..3847bf2 100644 >>>> --- a/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java >>>> +++ b/common/src/com/thoughtworks/go/config/materials/PackageMaterial.java >>>> @@ -168,6 +168,7 @@ public class PackageMaterial extends AbstractMaterial { >>>> @Override >>>> public void populateEnvironmentContext(EnvironmentVariableContext >>>> context, MaterialRevision materialRevision, File workingDir) { >>>> context.setProperty(upperCase(format("GO_PACKAGE_%s_LABEL", >>>> escapeEnvironmentVariable(getName().toString()))), >>>> materialRevision.getRevision().getRevision(), false); >>>> + context.setProperty(upperCase(format("GO_PACKAGE_%s_CHANGED", >>>> escapeEnvironmentVariable(getName().toString()))), >>>> Boolean.toString(materialRevision.isChanged()), false); >>>> for (ConfigurationProperty configurationProperty : >>>> getPackageDefinition().getRepository().getConfiguration()) { >>>> >>>> context.setProperty(getEnvironmentVariableKey("GO_REPO_%s_%s", >>>> configurationProperty.getConfigurationKey().getName()), >>>> configurationProperty.getValue(), >>>> configurationProperty.isSecure()); >>>> diff --git >>>> a/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java >>>> >>>> b/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java >>>> index 391bfae..1190cf9 100644 >>>> --- >>>> a/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java >>>> +++ >>>> b/common/src/com/thoughtworks/go/config/materials/PluggableSCMMaterial.java >>>> @@ -265,6 +265,7 @@ public class PluggableSCMMaterial extends >>>> AbstractMaterial { >>>> @Override >>>> public void populateEnvironmentContext(EnvironmentVariableContext >>>> context, MaterialRevision materialRevision, File workingDir) { >>>> context.setProperty(getEnvironmentVariableKey("GO_SCM_%s_%s", >>>> "LABEL"), materialRevision.getRevision().getRevision(), false); >>>> + context.setProperty(getEnvironmentVariableKey("GO_SCM_%s_%s", >>>> "CHANGED"), Boolean.toString(materialRevision.isChanged()), false); >>>> for (ConfigurationProperty configurationProperty : >>>> scmConfig.getConfiguration()) { >>>> context.setProperty(getEnvironmentVariableKey("GO_SCM_%s_%s", >>>> configurationProperty.getConfigurationKey().getName()), >>>> configurationProperty.getValue(), >>>> configurationProperty.isSecure()); >>>> diff --git >>>> a/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java >>>> b/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java >>>> index d3c86db..aa9509e 100644 >>>> --- a/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java >>>> +++ b/common/src/com/thoughtworks/go/config/materials/ScmMaterial.java >>>> @@ -129,13 +129,14 @@ public abstract class ScmMaterial extends >>>> AbstractMaterial { >>>> String toRevision = materialRevision.getRevision().getRevision(); >>>> String fromRevision = >>>> materialRevision.getOldestRevision().getRevision(); >>>> >>>> - setGoRevisionVariables(environmentVariableContext, fromRevision, >>>> toRevision); >>>> + setGoRevisionVariables(environmentVariableContext, >>>> materialRevision, fromRevision, toRevision); >>>> } >>>> >>>> - private void setGoRevisionVariables(EnvironmentVariableContext >>>> environmentVariableContext, String fromRevision, String toRevision) { >>>> + private void setGoRevisionVariables(EnvironmentVariableContext >>>> environmentVariableContext, MaterialRevision materialRevision, String >>>> fromRevision, String toRevision) { >>>> setVariableWithName(environmentVariableContext, toRevision, >>>> GO_REVISION); >>>> setVariableWithName(environmentVariableContext, toRevision, >>>> GO_TO_REVISION); >>>> setVariableWithName(environmentVariableContext, fromRevision, >>>> GO_FROM_REVISION); >>>> + setVariableWithName(environmentVariableContext, >>>> Boolean.toString(materialRevision.isChanged()), "GO_CHANGED"); >>>> } >>>> >>>> protected void setVariableWithName(EnvironmentVariableContext >>>> environmentVariableContext, String value, String propertyName) { >>>> diff --git >>>> a/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java >>>> >>>> b/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java >>>> index 016e049..ea3681a 100644 >>>> --- >>>> a/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java >>>> +++ >>>> b/common/src/com/thoughtworks/go/config/materials/dependency/DependencyMaterial.java >>>> @@ -160,6 +160,7 @@ public class DependencyMaterial extends >>>> AbstractMaterial { >>>> DependencyMaterialRevision revision = >>>> (DependencyMaterialRevision) materialRevision.getRevision(); >>>> context.setPropertyWithEscape(format("GO_DEPENDENCY_LABEL_%s", >>>> getName()), revision.getPipelineLabel()); >>>> context.setPropertyWithEscape(format("GO_DEPENDENCY_LOCATOR_%s", >>>> getName()), revision.getRevision()); >>>> + context.setPropertyWithEscape(format("GO_DEPENDENCY_CHANGED_%s", >>>> getName()), Boolean.toString(materialRevision.isChanged())); >>>> } >>>> >>>> public boolean isAutoUpdate() { >>>> >>>> >>>> >>>> >>>>> On Mon, Oct 3, 2016 at 6:01 AM, 'Anant Agarwal' via go-cd >>>>> <[email protected]> wrote: >>>>> Hi >>>>> >>>>> We have a bunch of micro services in separate code repositories. The >>>>> pipelines we have setup look like the diagram below. >>>>> >>>>> >>>>> At the moment whenever we make changes to any one service let's say >>>>> service1, integration test pipeline runs and then triggers deployment >>>>> of all services. >>>>> >>>>> What we want is that Integration test pipeline should run if changes to >>>>> any of the dependent services but it should trigger deployment only for >>>>> the particular service which originated the change. Is that possible? >>>>> >>>>> so we are kind of looking to achieve "AND" condition in pipeline trigger. >>>>> i.e. Deployment service 1 pipeline should trigger only when Service1 >>>>> Artefact was built AND integration test passed. and this should not >>>>> trigger Deployment service2 pipeline. >>>>> >>>>> >>>>> Appreciate your help. >>>>> >>>>> >>>>> Thanks >>>>> Anant Agarwal >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "go-cd" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to a topic in the Google > Groups "go-cd" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/go-cd/z1t0fy1Ldck/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
