Re: [openstack-dev] [OpenStack-Infra] Cross-Repo Dependencies in Zuul

2015-02-11 Thread Thierry Carrez
James E. Blair wrote:
> We have added support for cross-repo dependencies (CRD) in Zuul.  The
> important bits:
> 
> * To use them, include "Depends-On: " in the footer of
>   your commit message.  Use the full Change-ID ('I' + 40 characters).

Thanks for that!
I know that should make some StoryBoard devs quite happy.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [OpenStack-Infra] Cross-Repo Dependencies in Zuul

2015-02-10 Thread Boris Pavlovic
James,

Awesome! Amazing! You guys rock!=)

Best regards,
Boris Pavlovic



On Wed, Feb 11, 2015 at 1:26 AM, James E. Blair  wrote:

> Hi,
>
> We have added support for cross-repo dependencies (CRD) in Zuul.  The
> important bits:
>
> * To use them, include "Depends-On: " in the footer of
>   your commit message.  Use the full Change-ID ('I' + 40 characters).
>
> * These are one-way dependencies only -- do not create a cycle.
>
> * This is what all the grey dots and lines are in the check pipeline.
>
> Cross-Repo Dependencies Explained
> =
>
> There are two behaviors that might go by the name "cross-repo
> dependencies".  We call them one-way and multi-way.
>
> Multi-way CRD allow for bidirectional links between changes.  For
> instance, A depends on B and B depends on A.  The theory there is that
> both would be tested together and merged as a unit.  This is _not_ what
> we have implemented in Zuul.  Discussions over the past two years have
> revealed that this type of behavior could cause problems for continuous
> deploments as it means that two components must be upgraded
> simultaneously.  Not supporting this behavior is a choice we have made.
>
> One-way CRD behaves like a directed acyclic graph (DAG), like git
> itself, to indicate a one-way dependency relationship between changes in
> different git repos.  Change A may depend on B, but B may not depend on
> A.  This is what we have implemented in Zuul.
>
> Gate Pipeline
> =
>
> When Zuul sees CRD changes, it serializes them in the usual manner when
> enqueuing them into a pipeline.  This means that if change A depends on
> B, then when they are added to the gate pipeline, B will appear first
> and A will follow.  If tests for B fail, both B and A will be removed
> from the pipeline, and it will not be possible for A to merge until B
> does.
>
> Note that if changes with CRD do not share a change queue (such as the
> "integrated gate" then Zuul is unable to enqueue them together, and the
> first will be required to merge before the second is enqueued.
>
> Check Pipeline
> ==
>
> When changes are enqueued into the check pipeline, all of the related
> dependencies (both normal git-dependencies that come from parent commits
> as well as CRD changes) appear in a dependency graph, as in gate.  This
> means that even in the check pipeline, your change will be tested with
> its dependency.  So changes that were previously unable to be fully
> tested until a related change landed in a different repo may now be
> tested together from the start.
>
> All of the changes are still independent (so you will note that the
> whole pipeline does not share a graph as in gate), but for each change
> tested, all of its dependencies are visually connected to it, and they
> are used to construct the git references that Zuul uses when testing.
> When looking at this graph on the status page, you will note that the
> dependencies show up as grey dots, while the actual change tested shows
> up as red or green.  This is to indicate that the grey changes are only
> there to establish dependencies.  Even if one of the dependencies is
> also being tested, it will show up as a grey dot when used as a
> dependency, but separately and additionally will appear as its own red
> or green dot for its test.
>
> (If you don't see grey dots on the status page, reload the page to get
> the latest version.)
>
> Multiple Changes
> 
>
> A Gerrit change ID may refer to multiple changes (on multiple branches
> of the same project, or even multiple projects).  In these cases, Zuul
> will treat all of the changes with that change ID as dependencies.  So
> if you say that a tempest change Depends-On a change ID that has changes
> in nova master and nova stable/juno, then when testing the tempest
> change, both nova changes will be applied, and when deciding whether the
> tempest change can merge, both changes must merge ahead of it.
>
> A change may depend on more than one Gerrit change ID as well.  So it is
> possible for a change in tempest to depend on a change in devstack and a
> change in nova.  Simply add more "Depends-On:" lines to the footer.
>
> Cycles
> ==
>
> If a cycle is created by use of CRD, Zuul will abort its work very
> early.  There will be no message in Gerrit and no changes that are part
> of the cycle will be enqueued into any pipeline.  This is to protect
> Zuul from infinite loops.  I hope that we can improve this to at least
> leave a message in Gerrit in the future.  But in the meantime, please be
> cognizant of this and do not create dependency cycles with Depends-On
> lines.
>
> Examples
> 
>
> The following two infra changes have been tested together because of the
> Depends-On: line in the commit message of the first:
>
>   https://review.openstack.org/#/c/152508/
>   https://review.openstack.org/#/c/152504/
>
> In fact, you can see earlier test results failing until it was rechecked
>