On Monday, 25 July 2016 08:50:07 UTC+2, Varsha Varadarajan wrote:
>
> The scenario:
>>
>>
>> - A git repository "repo" with "core" and "app" root paths.
>> - A "BuildCore" pipeline that uses "repo" with a whitelist on
>> "core/*" - triggers automatically
>> - A "BuildApp" pipeline that uses "repo" with a blacklist on "core/*"
>> and an upstream pipeline dependency / material on "BuildCore" - triggers
>> automatically
>>
>> The bug is that it seems like Go traces its upstream pipelines' materials
>> and if a common material is found, the downstream pipeline will wait for
>> the upstream pipeline to finish before triggering.
>>
>>
>>
> Given that the pipeline BuildApp is dependent on the git repository 'repo'
> and pipeline 'BuildCore', any *valid* change in either materials will
> trigger 'BuildApp'. In this scenario, a valid change is any change to
> 'core/*'.
> If you observe the build cause for BuildApp, when a commit is made in
> 'core/*', the build cause will only list the 'BuildCore' as the reason for
> the trigger.
> It may seem that it is a violation of the blacklist in BuildApp, but a
> change in core/* is not the direct reason for BuildApp being triggered. It
> is the expected behaviour.
>
I know it is the expected behaviour, but this behaviour could be improved
to better suit build requirements.
If I were to move the "core" and "app" paths to different repos, with
"core" repo at revision "c01" and "app" repo at revision "a01", then I have
a nice workflow in Go, the way it was designed:
- If I commit "#c02" to the "core" repo, BuildCore will trigger. The new
"#c02" build of BuildCore will trigger BuildApp with the old "#a01"
revision from "app".
- If I commit "#a02" to the "app" repo, the AppBuild pipeline will build
"#a02" with "#c02". Subsequent commits to the app repo will build "#a03",
"#a04", etc with "#c02"
This workflow is not possible when using paths in 1 repo for different
pipelines. I'm suggesting this is a shortfall in the workflow design -
something that could be improved to give the same flow in a monolithic repo
structure. The fan-in algorithm should not only trace materials, but also
factor in black / whitelists as part of the algorithm.
I am able to work around this and get the desired workflow by using a
different hostname for the same repo in the 2 pipelines.
>
> In this scenario, if a commit is pushed to "app/foo", both BuildCore and
>> BuildApp detect the new "repo" material. BuildCore doesn't trigger because
>> only "core" is in its whitelist ("app/foo" is blacklisted). BuildApp finds
>> the change, but doesn't trigger because it sees that BuildCore also uses
>> the "repo" material and wants to wait for it to trigger and finish - it
>> doesn't realize that BuildCore will not trigger due to the blacklist -
>> BuildApp never triggers automatically. I'd imagine that it's only when I
>> push to "core/bar" that BuildCore would trigger and that would then kick
>> off BuildApp...
>>
>
> Since BuildApp is dependent on BuildCore and git repository 'repo', the
> fan-in algorithm kicks in where it tries to find a compatible revision with
> both the git material 'repo' and BuildCore. Since it cannot find that
> BuildCore has been run with "app/foo", BuildApp will never get triggered.
> This behaviour exists irrespective of the blacklist and whitelist feature.
> And, because of the fan-in algorithm, the only valid change for BuildApp,
> as mentioned above to be triggered is any change to 'core/*' only.
>
>
>>
>> I now either need to remove the black / white lists and build BuildCore
>> for all commits or trigger manually. I'd imagine a solution to this is that
>> BuildApp should not only check if "repo" is an upstream of "BuildCore" but
>> should also ensure that the new "repo" material is not blacklisted there....
>>
>
> Yes, that is correct.
> The whitelist on the upstream pipeline "BuildCore" is respected even in
> BuildApp because of the dependency. A workaround would be to remove the
> whitelist in "BuildCore". However, keep in mind once the whitelist is
> removed, the blacklist in downstream "BuildApp" will not be respected (that
> is, blacklist on "BuildApp" is useless in this scenario to have) as any
> change triggering "BuildCore" will end up triggering "BuildApp" anyway.
>
> Thank You.
> Varsha
>
--
You received this message because you are subscribed to the Google Groups
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
