Hi Joel,
I think the missing piece may be that *#{UPSTREAM_BUILD_PIPELINE} *does not
need to be a direct parent of the Pipeline where the Fetch Artifact Task
resides.
I recommend the following changes to Pipeline C:
1. Change the value of *#{UPSTREAM_BUILD_PIPELINE}* from *Pipeline A* to
*Pipeline
A/Pipeline B*.
2. Remove the *Pipeline A* Material.
I modified your original XML document to show what I mean.
Hope this helps,
Jason
On Friday, 27 November 2020 at 10:43:15 UTC-5 [email protected] wrote:
> Thank you for your reply, Ketan.
>
> I've come across the fan-in & fan-out capabilities - which is one of the
> main reasons I decided to give GoCD a try and hopefully switch to it
> eventually.
> I've read the documentation you've linked prior to creating this
> conversation. However, I'm unclear as to how to modify my pipeline C (or
> rather template T_C) so there's no more explicit dependency on the upstream
> pipeline. In my opinion I did this by having the *Fetch Artifact* task in
> T_C which allows C which uses T_C to specify the upstream pipeline with
> *#{UPSTREAM_BUILD_PIPELINE}*.
> What am I missing / doing wrong?
>
> In my pipeline C -> Materials I am unable to remove the dependency to the
> upstream build pipeline A. As far as I can tell this has been added
> automatically when creating pipeline A from template T_A as T_A requires
> the *#{UPSTREAM_BUILD_PIPELINE}* parameter for the *Fetch Artifact* task.
>
> ~ Joel
>
>
> On Friday, November 27, 2020 at 4:16:38 PM UTC+1 [email protected]
> wrote:
>
>> If pipeline "C" requires a binary from "A", there's no need to add an
>> explicit dependency. You can specify an upstream pipeline as part of your
>> fetch artifact task.
>>
>> In either case, C should not trigger twice. Pipeline C should wait for B
>> to be complete before kicking off.
>>
>> Also — what you're looking at is called "fan-in" in gocd terminology. This
>> document <https://docs.gocd.org/current/advanced_usage/fan_in.html>
>> might help.
>>
>> - Ketan
>>
>>
>>
>> On Fri, Nov 27, 2020 at 7:47 PM Joel Bodenmann <[email protected]> wrote:
>>
>>> Hello folks,
>>>
>>> Consider the following scenario:
>>>
>>> - Template *T_A*: Builds some binaries and stores them as an
>>> artifact.
>>> - Template *T_B*: Fetches binaries from the upstream pipeline *T_A *and
>>> runs tests. To achieve this, *T_B* has a *Fetch Artifact *task with
>>> the upstream pipeline as a parameter *#{UPSTREAM_BUILD_PIPELINE}.*
>>> - Template *T_C*: Fetches binaries from upstream pipeline *T_A* and
>>> creates packages. To achieve this, *T_C* has a *Fetch Artifact *task
>>> with the upstream pipeline as a parameter
>>> *#{UPSTREAM_BUILD_PIPELINE}.*
>>> - Pipeline *A*: Uses template *T_A*.
>>> - Pipeline *B*: Uses template *T_B**. *Sets *UPSTREAM_BUILD_PIPELINE
>>> *to pipeline *A.*
>>> - Pipeline *C*: Uses template *T_C*. Sets *UPSTREAM_BUILD_PIPELINE *
>>> to pipeline *A.*
>>>
>>> This results in the following VSM:
>>>
>>>
>>> The problem I have is that pipeline C will be triggered twice: Once
>>> after A finishes and once after B finishes. The expected behavior (or the
>>> behavior I wish to achieve) is that C will only run after B finishes
>>> successfully.
>>>
>>> How can I make sure that C only runs after successful completion of B
>>> (and therefore implicitly A)?
>>>
>>> Attached is the corresponding XML fragment of my templates & pipelines
>>> where:
>>>
>>> - *T_A -> Build-CMake-OS-Compiler*
>>> - *T_B -> Test-OS*
>>> - *T_C -> Package-OS*
>>> - *A -> GPDS-Build-FreeBSD-GCC*
>>> - *B -> GPDS-Test-FreeBSD-GCC*
>>> - *C -> GPDS-Package-FreeBSD*
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/0a8a6dc4-0b49-4036-a573-44d0feca32b9n%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/go-cd/0a8a6dc4-0b49-4036-a573-44d0feca32b9n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/go-cd/b3e4e221-1a00-424a-9d51-fef13e885b7fn%40googlegroups.com.
<pipelines group="GPDS">
<pipeline name="GPDS-Build-FreeBSD-GCC" template="Build-CMake-OS-Compiler">
<params>
<param name="COMPILER_SUITE">gcc</param>
<param name="OS">FreeBSD</param>
<param name="CMAKE_CXX_COMPILER">g++10</param>
<param name="CMAKE_GENERATOR">Unix Makefiles</param>
<param name="CMAKE_C_COMPILER">gcc10</param>
<param name="NUM_JOBS">4</param>
</params>
<materials>
<git url="https://github.com/simulton/gpds"; />
</materials>
</pipeline>
<pipeline name="GPDS-Test-FreeBSD-GCC" template="Test-OS">
<params>
<param name="TEST_BIN">build/test/gpds-tests</param>
<param name="OS">FreeBSD</param>
<param name="UPSTREAM_BUILD_PIPELINE">GPDS-Build-FreeBSD-GCC</param>
</params>
<materials>
<pipeline pipelineName="GPDS-Build-FreeBSD-GCC" stageName="Build" />
</materials>
</pipeline>
<pipeline name="GPDS-Package-FreeBSD" template="Package-OS">
<params>
<param name="UPSTREAM_BUILD_PIPELINE">GPDS-Build-FreeBSD-GCC/GPDS-Test-FreeBSD-GCC</param>
<param name="OS">FreeBSD</param>
</params>
<materials>
<pipeline pipelineName="GPDS-Test-FreeBSD-GCC" stageName="Run" />
</materials>
</pipeline>
</pipelines>
<templates>
<pipeline name="Build-CMake-OS-Compiler">
<stage name="Build">
<jobs>
<job name="Build">
<tasks>
<exec command="mkdir">
<arg>build</arg>
<runif status="passed" />
</exec>
<exec command="cmake" workingdir="build">
<arg>-G</arg>
<arg>#{CMAKE_GENERATOR}</arg>
<arg>-DCMAKE_C_COMPILER=#{CMAKE_C_COMPILER}</arg>
<arg>-DCMAKE_CXX_COMPILER=#{CMAKE_CXX_COMPILER}</arg>
<arg>..</arg>
<runif status="passed" />
</exec>
<exec command="make" workingdir="build">
<arg>-j#{NUM_JOBS}</arg>
<runif status="passed" />
</exec>
</tasks>
<resources>
<resource>#{OS}</resource>
<resource>#{COMPILER_SUITE}</resource>
</resources>
<artifacts>
<artifact type="build" src="build" />
</artifacts>
</job>
</jobs>
</stage>
</pipeline>
<pipeline name="Test-OS">
<stage name="Run">
<approval type="success" allowOnlyOnSuccess="true" />
<jobs>
<job name="Run">
<tasks>
<fetchartifact artifactOrigin="gocd" srcdir="build" pipeline="#{UPSTREAM_BUILD_PIPELINE}" stage="Build" job="Build">
<runif status="passed" />
</fetchartifact>
<exec command="chmod">
<arg>+x</arg>
<arg>#{TEST_BIN}</arg>
<runif status="passed" />
</exec>
<exec command="#{TEST_BIN}">
<runif status="passed" />
</exec>
</tasks>
<resources>
<resource>#{OS}</resource>
</resources>
</job>
</jobs>
</stage>
</pipeline>
<pipeline name="Package-OS">
<stage name="Generate">
<jobs>
<job name="Installer">
<tasks>
<fetchartifact artifactOrigin="gocd" srcdir="build" pipeline="#{UPSTREAM_BUILD_PIPELINE}" stage="Build" job="Build">
<runif status="passed" />
</fetchartifact>
<exec command="ls" workingdir="build">
<arg>-lhs</arg>
<runif status="passed" />
</exec>
</tasks>
<resources>
<resource>#{OS}</resource>
</resources>
</job>
</jobs>
</stage>
</pipeline>
</templates>
