Re: [openstack-dev] Cross-Repo Dependencies in Zuul
Wow, this is great! Thank you!
Chris
On Wed, Feb 11, 2015 at 8:56 AM, James E. Blair wrote:
> Hi,
>
> We have added support for cross-repo dependencies (CRD) in Zuul. The
> important bits:
>
> * To use them, include "Depends-On: " in the footer of
> your commit message. Use the full Change-ID ('I' + 40 characters).
>
> * These are one-way dependencies only -- do not create a cycle.
>
> * This is what all the grey dots and lines are in the check pipeline.
>
> Cross-Repo Dependencies Explained
> =
>
> There are two behaviors that might go by the name "cross-repo
> dependencies". We call them one-way and multi-way.
>
> Multi-way CRD allow for bidirectional links between changes. For
> instance, A depends on B and B depends on A. The theory there is that
> both would be tested together and merged as a unit. This is _not_ what
> we have implemented in Zuul. Discussions over the past two years have
> revealed that this type of behavior could cause problems for continuous
> deploments as it means that two components must be upgraded
> simultaneously. Not supporting this behavior is a choice we have made.
>
> One-way CRD behaves like a directed acyclic graph (DAG), like git
> itself, to indicate a one-way dependency relationship between changes in
> different git repos. Change A may depend on B, but B may not depend on
> A. This is what we have implemented in Zuul.
>
> Gate Pipeline
> =
>
> When Zuul sees CRD changes, it serializes them in the usual manner when
> enqueuing them into a pipeline. This means that if change A depends on
> B, then when they are added to the gate pipeline, B will appear first
> and A will follow. If tests for B fail, both B and A will be removed
> from the pipeline, and it will not be possible for A to merge until B
> does.
>
> Note that if changes with CRD do not share a change queue (such as the
> "integrated gate" then Zuul is unable to enqueue them together, and the
> first will be required to merge before the second is enqueued.
>
> Check Pipeline
> ==
>
> When changes are enqueued into the check pipeline, all of the related
> dependencies (both normal git-dependencies that come from parent commits
> as well as CRD changes) appear in a dependency graph, as in gate. This
> means that even in the check pipeline, your change will be tested with
> its dependency. So changes that were previously unable to be fully
> tested until a related change landed in a different repo may now be
> tested together from the start.
>
> All of the changes are still independent (so you will note that the
> whole pipeline does not share a graph as in gate), but for each change
> tested, all of its dependencies are visually connected to it, and they
> are used to construct the git references that Zuul uses when testing.
> When looking at this graph on the status page, you will note that the
> dependencies show up as grey dots, while the actual change tested shows
> up as red or green. This is to indicate that the grey changes are only
> there to establish dependencies. Even if one of the dependencies is
> also being tested, it will show up as a grey dot when used as a
> dependency, but separately and additionally will appear as its own red
> or green dot for its test.
>
> (If you don't see grey dots on the status page, reload the page to get
> the latest version.)
>
> Multiple Changes
>
>
> A Gerrit change ID may refer to multiple changes (on multiple branches
> of the same project, or even multiple projects). In these cases, Zuul
> will treat all of the changes with that change ID as dependencies. So
> if you say that a tempest change Depends-On a change ID that has changes
> in nova master and nova stable/juno, then when testing the tempest
> change, both nova changes will be applied, and when deciding whether the
> tempest change can merge, both changes must merge ahead of it.
>
> A change may depend on more than one Gerrit change ID as well. So it is
> possible for a change in tempest to depend on a change in devstack and a
> change in nova. Simply add more "Depends-On:" lines to the footer.
>
> Cycles
> ==
>
> If a cycle is created by use of CRD, Zuul will abort its work very
> early. There will be no message in Gerrit and no changes that are part
> of the cycle will be enqueued into any pipeline. This is to protect
> Zuul from infinite loops. I hope that we can improve this to at least
> leave a message in Gerrit in the future. But in the meantime, please be
> cognizant of this and do not create dependency cycles with Depends-On
> lines.
>
> Examples
>
>
> The following two infra changes have been tested together because of the
> Depends-On: line in the commit message of the first:
>
> https://review.openstack.org/#/c/152508/
> https://review.openstack.org/#/c/152504/
>
> In fact, you can see earlier test results failing until it was rechecked
> after CRD went into production (aroun
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
"Sullivan, Jon Paul" writes: >> A change may depend on more than one Gerrit change ID as well. So it is >> possible for a change in tempest to depend on a change in devstack and a >> change in nova. Simply add more "Depends-On:" lines to the footer. > > Have you considered a case where changes A and C are in nova, and > change B is in neutron such that A depends on B depends on C? > > And does that create a similar, and uncaught, situation as per the > Multi-way CRD, where two components will need to be updated > simultaneously? That depends. From the info you provided, I see: A(nova) -> B(neutron) -> C(nova) And there is no cycle, that just means that they are enqueued as C,B,A, and the git checkouts for those tests will be: C: nova branch tip+C B: nova branch tip+C, neutron branch tip+B A: nova branch tip+C+A, neutron branch tip+B However, if A is a git parent of C, then there would be a cycle: A(nova) -> B(neutron) -> C(nova) -> A(nova) So Zuul would stop processing that. -Jim __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
Thanks, most awesome. Alas, if this was around three weeks ago, I'd still have a full head of hair. Now all I have is a lifetime of savings on hairdressing. -amrith | -Original Message- | From: James E. Blair [mailto:cor...@inaugust.com] | Sent: Wednesday, February 11, 2015 11:33 AM | To: OpenStack Development Mailing List (not for usage questions) | Cc: openstack-in...@lists.openstack.org | Subject: Re: [openstack-dev] Cross-Repo Dependencies in Zuul | | Amrith Kumar writes: | | > This is awesome news! | > | > If I understand correctly, your description of "multiple changes" | > describes cases where a single change depends on multiple changes. My | > question is related to one-to-one dependencies in the form A -> B -> | > C. | > | > I'm assuming that this is supported as well in the case where, for | > example, A and C are in one repository and B is in another repository, | > yes? | | Yes and yes. When writing the "multiple changes" section, I was | describing how you can make A->B and A->C. If you're curious, that gets | serialized as B,C,A. | | A->B->C works as well (and of course is serialized as C,B,A). The | enqueuing function is recursive, so, well, it works up to the Python stack | limit. :) | | -Jim | | __ | OpenStack Development Mailing List (not for usage questions) | Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe | http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
Amrith Kumar writes: > This is awesome news! > > If I understand correctly, your description of "multiple changes" > describes cases where a single change depends on multiple changes. My > question is related to one-to-one dependencies in the form A -> B -> > C. > > I'm assuming that this is supported as well in the case where, for > example, A and C are in one repository and B is in another repository, > yes? Yes and yes. When writing the "multiple changes" section, I was describing how you can make A->B and A->C. If you're curious, that gets serialized as B,C,A. A->B->C works as well (and of course is serialized as C,B,A). The enqueuing function is recursive, so, well, it works up to the Python stack limit. :) -Jim __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
This is awesome news!
If I understand correctly, your description of "multiple changes" describes
cases where a single change depends on multiple changes. My question is related
to one-to-one dependencies in the form A -> B -> C.
I'm assuming that this is supported as well in the case where, for example, A
and C are in one repository and B is in another repository, yes?
Thanks,
-amrith
| -Original Message-
| From: James E. Blair [mailto:cor...@inaugust.com]
| Sent: Tuesday, February 10, 2015 5:26 PM
| To: OpenStack Development Mailing List
| Cc: openstack-in...@lists.openstack.org
| Subject: [openstack-dev] Cross-Repo Dependencies in Zuul
|
| Hi,
|
| We have added support for cross-repo dependencies (CRD) in Zuul. The
| important bits:
|
| * To use them, include "Depends-On: " in the footer of
| your commit message. Use the full Change-ID ('I' + 40 characters).
|
| * These are one-way dependencies only -- do not create a cycle.
|
| * This is what all the grey dots and lines are in the check pipeline.
|
| Cross-Repo Dependencies Explained
| =
|
| There are two behaviors that might go by the name "cross-repo
| dependencies". We call them one-way and multi-way.
|
| Multi-way CRD allow for bidirectional links between changes. For
| instance, A depends on B and B depends on A. The theory there is that
| both would be tested together and merged as a unit. This is _not_ what we
| have implemented in Zuul. Discussions over the past two years have
| revealed that this type of behavior could cause problems for continuous
| deploments as it means that two components must be upgraded
| simultaneously. Not supporting this behavior is a choice we have made.
|
| One-way CRD behaves like a directed acyclic graph (DAG), like git itself,
| to indicate a one-way dependency relationship between changes in different
| git repos. Change A may depend on B, but B may not depend on A. This is
| what we have implemented in Zuul.
|
| Gate Pipeline
| =
|
| When Zuul sees CRD changes, it serializes them in the usual manner when
| enqueuing them into a pipeline. This means that if change A depends on B,
| then when they are added to the gate pipeline, B will appear first and A
| will follow. If tests for B fail, both B and A will be removed from the
| pipeline, and it will not be possible for A to merge until B does.
|
| Note that if changes with CRD do not share a change queue (such as the
| "integrated gate" then Zuul is unable to enqueue them together, and the
| first will be required to merge before the second is enqueued.
|
| Check Pipeline
| ==
|
| When changes are enqueued into the check pipeline, all of the related
| dependencies (both normal git-dependencies that come from parent commits
| as well as CRD changes) appear in a dependency graph, as in gate. This
| means that even in the check pipeline, your change will be tested with its
| dependency. So changes that were previously unable to be fully tested
| until a related change landed in a different repo may now be tested
| together from the start.
|
| All of the changes are still independent (so you will note that the whole
| pipeline does not share a graph as in gate), but for each change tested,
| all of its dependencies are visually connected to it, and they are used to
| construct the git references that Zuul uses when testing.
| When looking at this graph on the status page, you will note that the
| dependencies show up as grey dots, while the actual change tested shows up
| as red or green. This is to indicate that the grey changes are only there
| to establish dependencies. Even if one of the dependencies is also being
| tested, it will show up as a grey dot when used as a dependency, but
| separately and additionally will appear as its own red or green dot for
| its test.
|
| (If you don't see grey dots on the status page, reload the page to get the
| latest version.)
|
| Multiple Changes
|
|
| A Gerrit change ID may refer to multiple changes (on multiple branches of
| the same project, or even multiple projects). In these cases, Zuul will
| treat all of the changes with that change ID as dependencies. So if you
| say that a tempest change Depends-On a change ID that has changes in nova
| master and nova stable/juno, then when testing the tempest change, both
| nova changes will be applied, and when deciding whether the tempest change
| can merge, both changes must merge ahead of it.
|
| A change may depend on more than one Gerrit change ID as well. So it is
| possible for a change in tempest to depend on a change in devstack and a
| change in nova. Simply add more "Depends-On:" lines to the footer.
|
| Cycles
| ==
|
| If a cycle is created by use of CRD, Zuul will abort its work very early.
| There will be no message in Gerrit and no changes that are part
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
Firstly, this is a great feature - thanks for getting it implemented! > -Original Message- > From: James E. Blair [mailto:cor...@inaugust.com] > Sent: 10 February 2015 22:26 > To: OpenStack Development Mailing List > Cc: openstack-in...@lists.openstack.org > Subject: [openstack-dev] Cross-Repo Dependencies in Zuul > : snip : > > Cross-Repo Dependencies Explained > = > > There are two behaviors that might go by the name "cross-repo > dependencies". We call them one-way and multi-way. > > Multi-way CRD allow for bidirectional links between changes. For > instance, A depends on B and B depends on A. The theory there is that > both would be tested together and merged as a unit. This is _not_ what > we have implemented in Zuul. Discussions over the past two years have > revealed that this type of behavior could cause problems for continuous > deploments as it means that two components must be upgraded > simultaneously. Not supporting this behavior is a choice we have made. > > One-way CRD behaves like a directed acyclic graph (DAG), like git > itself, to indicate a one-way dependency relationship between changes in > different git repos. Change A may depend on B, but B may not depend on > A. This is what we have implemented in Zuul. > : snip : > > Multiple Changes > > : snip : > > A change may depend on more than one Gerrit change ID as well. So it is > possible for a change in tempest to depend on a change in devstack and a > change in nova. Simply add more "Depends-On:" lines to the footer. Have you considered a case where changes A and C are in nova, and change B is in neutron such that A depends on B depends on C? And does that create a similar, and uncaught, situation as per the Multi-way CRD, where two components will need to be updated simultaneously? : snip : > > -Jim > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev- > requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Thanks, Jon-Paul Sullivan ☺ Cloud Services - @hpcloud Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway. Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2. Registered Number: 361933 The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL". __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
OH MY GOD!
I'll throw a party! Thanks to everyone involved in making this
happen.
Flavio
On 10/02/15 14:26 -0800, James E. Blair wrote:
Hi,
We have added support for cross-repo dependencies (CRD) in Zuul. The
important bits:
* To use them, include "Depends-On: " in the footer of
your commit message. Use the full Change-ID ('I' + 40 characters).
* These are one-way dependencies only -- do not create a cycle.
* This is what all the grey dots and lines are in the check pipeline.
Cross-Repo Dependencies Explained
=
There are two behaviors that might go by the name "cross-repo
dependencies". We call them one-way and multi-way.
Multi-way CRD allow for bidirectional links between changes. For
instance, A depends on B and B depends on A. The theory there is that
both would be tested together and merged as a unit. This is _not_ what
we have implemented in Zuul. Discussions over the past two years have
revealed that this type of behavior could cause problems for continuous
deploments as it means that two components must be upgraded
simultaneously. Not supporting this behavior is a choice we have made.
One-way CRD behaves like a directed acyclic graph (DAG), like git
itself, to indicate a one-way dependency relationship between changes in
different git repos. Change A may depend on B, but B may not depend on
A. This is what we have implemented in Zuul.
Gate Pipeline
=
When Zuul sees CRD changes, it serializes them in the usual manner when
enqueuing them into a pipeline. This means that if change A depends on
B, then when they are added to the gate pipeline, B will appear first
and A will follow. If tests for B fail, both B and A will be removed
from the pipeline, and it will not be possible for A to merge until B
does.
Note that if changes with CRD do not share a change queue (such as the
"integrated gate" then Zuul is unable to enqueue them together, and the
first will be required to merge before the second is enqueued.
Check Pipeline
==
When changes are enqueued into the check pipeline, all of the related
dependencies (both normal git-dependencies that come from parent commits
as well as CRD changes) appear in a dependency graph, as in gate. This
means that even in the check pipeline, your change will be tested with
its dependency. So changes that were previously unable to be fully
tested until a related change landed in a different repo may now be
tested together from the start.
All of the changes are still independent (so you will note that the
whole pipeline does not share a graph as in gate), but for each change
tested, all of its dependencies are visually connected to it, and they
are used to construct the git references that Zuul uses when testing.
When looking at this graph on the status page, you will note that the
dependencies show up as grey dots, while the actual change tested shows
up as red or green. This is to indicate that the grey changes are only
there to establish dependencies. Even if one of the dependencies is
also being tested, it will show up as a grey dot when used as a
dependency, but separately and additionally will appear as its own red
or green dot for its test.
(If you don't see grey dots on the status page, reload the page to get
the latest version.)
Multiple Changes
A Gerrit change ID may refer to multiple changes (on multiple branches
of the same project, or even multiple projects). In these cases, Zuul
will treat all of the changes with that change ID as dependencies. So
if you say that a tempest change Depends-On a change ID that has changes
in nova master and nova stable/juno, then when testing the tempest
change, both nova changes will be applied, and when deciding whether the
tempest change can merge, both changes must merge ahead of it.
A change may depend on more than one Gerrit change ID as well. So it is
possible for a change in tempest to depend on a change in devstack and a
change in nova. Simply add more "Depends-On:" lines to the footer.
Cycles
==
If a cycle is created by use of CRD, Zuul will abort its work very
early. There will be no message in Gerrit and no changes that are part
of the cycle will be enqueued into any pipeline. This is to protect
Zuul from infinite loops. I hope that we can improve this to at least
leave a message in Gerrit in the future. But in the meantime, please be
cognizant of this and do not create dependency cycles with Depends-On
lines.
Examples
The following two infra changes have been tested together because of the
Depends-On: line in the commit message of the first:
https://review.openstack.org/#/c/152508/
https://review.openstack.org/#/c/152504/
In fact, you can see earlier test results failing until it was rechecked
after CRD went into production (around 2015-02-10 15:20 UTC).
This devstack change depended on a grenade change:
https://review.openstack.org/#/c/154575/
https://review.openstack.org/
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
On 2015-02-11 00:33:03 +0100 (+0100), Ihar Hrachyshka wrote: > Long term, I can't agree more. Short term, while we decouple stuff > and in the middle of the process, that's too idealistic to be > true. [...] By implementing features to support the way we all generally agree things _should_ be rather than making it easier to leave things the way they _shouldn't_ we help encourage sane design patterns. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 02/11/2015 12:09 AM, Sean Dague wrote:
> On 02/10/2015 05:47 PM, Ihar Hrachyshka wrote:
>> So cool! A comment inline.
>>
>> On 02/10/2015 11:26 PM, James E. Blair wrote:
>>> Hi,
>>
>>> We have added support for cross-repo dependencies (CRD) in
>>> Zuul. The important bits:
>>
>>> * To use them, include "Depends-On: " in the
>>> footer of your commit message. Use the full Change-ID ('I' +
>>> 40 characters).
>>
>>> * These are one-way dependencies only -- do not create a
>>> cycle.
>>
>>> * This is what all the grey dots and lines are in the check
>>> pipeline.
>>
>>> Cross-Repo Dependencies Explained
>>> =
>>
>>> There are two behaviors that might go by the name "cross-repo
>>> dependencies". We call them one-way and multi-way.
>>
>>> Multi-way CRD allow for bidirectional links between changes.
>>> For instance, A depends on B and B depends on A. The theory
>>> there is that both would be tested together and merged as a
>>> unit. This is _not_ what we have implemented in Zuul.
>>> Discussions over the past two years have revealed that this
>>> type of behavior could cause problems for continuous deploments
>>> as it means that two components must be upgraded
>>> simultaneously. Not supporting this behavior is a choice we
>>> have made.
>>
>> Though I understand your reasoning behind not providing atomic
>> merges of multiple commits, I want to say that the feature would
>> be very useful for projects that are not decoupled enough. F.e.
>> see neutron advanced services repos that are split out of main
>> neutron project but still use lots of code from neutron
>> namespace, making it hard to introduce changes into the main repo
>> without breaking advanced services for some time. The same is
>> true for decomposed plugins and mechanism drivers for neutron.
>
> Atomic merges across git trees is a really bad idea.
>
> If you want atomic merges, there is an easy solution, be one git
> tree.
>
> If you are allowing folks externally to consume your code directly
> via import, then that needs to be a stable interface that has
> deprecation rules associated with it. All else is craziness.
Long term, I can't agree more. Short term, while we decouple stuff and
in the middle of the process, that's too idealistic to be true.
Though we think about stabilizing API, moving it to a separate common
library, and even spinning up advanced services into separate services
with their own REST. It will just take some time.
/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v1
iQEcBAEBAgAGBQJU2pUvAAoJEC5aWaUY1u57+KUH/1j3Yl4fdxGJcCIy26s3otSe
DIYopsOiQ0jb2XQ9ZlJ59Kq/KsYKUnHY46Pn6ZBH+iBBbjJke/VT8RaPZ9dJbI55
seTCi/ODEz1fYoZrhlQxRVKm7H465dBYffGKjp6eNfmL6DnIjBnvFcrL2dmKdSkV
BDGmiGBXaNJ7CDBH1PEdFoHpL7zn8ToPxgaEVEU2AH9WjIyX6gO5jYpDKvMWjvJt
O99WSY3MNVkIveFIyiISswXkJK5Bieif/DFmSlWpMKETiT/vRePjKuY0GlDKqC/r
yjKAEbC63Hu3Q0HUMJPc99XZM7TKJ5+I9krFuTw9zBy/ZfxUay5aBdgVzu65j8g=
=63NO
-END PGP SIGNATURE-
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
On 02/10/2015 05:47 PM, Ihar Hrachyshka wrote:
> So cool! A comment inline.
>
> On 02/10/2015 11:26 PM, James E. Blair wrote:
>> Hi,
>
>> We have added support for cross-repo dependencies (CRD) in Zuul.
>> The important bits:
>
>> * To use them, include "Depends-On: " in the
>> footer of your commit message. Use the full Change-ID ('I' + 40
>> characters).
>
>> * These are one-way dependencies only -- do not create a cycle.
>
>> * This is what all the grey dots and lines are in the check
>> pipeline.
>
>> Cross-Repo Dependencies Explained
>> =
>
>> There are two behaviors that might go by the name "cross-repo
>> dependencies". We call them one-way and multi-way.
>
>> Multi-way CRD allow for bidirectional links between changes. For
>> instance, A depends on B and B depends on A. The theory there is
>> that both would be tested together and merged as a unit. This is
>> _not_ what we have implemented in Zuul. Discussions over the past
>> two years have revealed that this type of behavior could cause
>> problems for continuous deploments as it means that two components
>> must be upgraded simultaneously. Not supporting this behavior is a
>> choice we have made.
>
> Though I understand your reasoning behind not providing atomic merges
> of multiple commits, I want to say that the feature would be very
> useful for projects that are not decoupled enough. F.e. see neutron
> advanced services repos that are split out of main neutron project but
> still use lots of code from neutron namespace, making it hard to
> introduce changes into the main repo without breaking advanced
> services for some time. The same is true for decomposed plugins and
> mechanism drivers for neutron.
Atomic merges across git trees is a really bad idea.
If you want atomic merges, there is an easy solution, be one git tree.
If you are allowing folks externally to consume your code directly via
import, then that needs to be a stable interface that has deprecation
rules associated with it. All else is craziness.
-Sean
--
Sean Dague
http://dague.net
signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cross-Repo Dependencies in Zuul
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
So cool! A comment inline.
On 02/10/2015 11:26 PM, James E. Blair wrote:
> Hi,
>
> We have added support for cross-repo dependencies (CRD) in Zuul.
> The important bits:
>
> * To use them, include "Depends-On: " in the
> footer of your commit message. Use the full Change-ID ('I' + 40
> characters).
>
> * These are one-way dependencies only -- do not create a cycle.
>
> * This is what all the grey dots and lines are in the check
> pipeline.
>
> Cross-Repo Dependencies Explained
> =
>
> There are two behaviors that might go by the name "cross-repo
> dependencies". We call them one-way and multi-way.
>
> Multi-way CRD allow for bidirectional links between changes. For
> instance, A depends on B and B depends on A. The theory there is
> that both would be tested together and merged as a unit. This is
> _not_ what we have implemented in Zuul. Discussions over the past
> two years have revealed that this type of behavior could cause
> problems for continuous deploments as it means that two components
> must be upgraded simultaneously. Not supporting this behavior is a
> choice we have made.
Though I understand your reasoning behind not providing atomic merges
of multiple commits, I want to say that the feature would be very
useful for projects that are not decoupled enough. F.e. see neutron
advanced services repos that are split out of main neutron project but
still use lots of code from neutron namespace, making it hard to
introduce changes into the main repo without breaking advanced
services for some time. The same is true for decomposed plugins and
mechanism drivers for neutron.
>
> One-way CRD behaves like a directed acyclic graph (DAG), like git
> itself, to indicate a one-way dependency relationship between
> changes in different git repos. Change A may depend on B, but B
> may not depend on A. This is what we have implemented in Zuul.
>
> Gate Pipeline =
>
> When Zuul sees CRD changes, it serializes them in the usual manner
> when enqueuing them into a pipeline. This means that if change A
> depends on B, then when they are added to the gate pipeline, B will
> appear first and A will follow. If tests for B fail, both B and A
> will be removed from the pipeline, and it will not be possible for
> A to merge until B does.
>
> Note that if changes with CRD do not share a change queue (such as
> the "integrated gate" then Zuul is unable to enqueue them together,
> and the first will be required to merge before the second is
> enqueued.
>
> Check Pipeline ==
>
> When changes are enqueued into the check pipeline, all of the
> related dependencies (both normal git-dependencies that come from
> parent commits as well as CRD changes) appear in a dependency
> graph, as in gate. This means that even in the check pipeline,
> your change will be tested with its dependency. So changes that
> were previously unable to be fully tested until a related change
> landed in a different repo may now be tested together from the
> start.
>
> All of the changes are still independent (so you will note that
> the whole pipeline does not share a graph as in gate), but for each
> change tested, all of its dependencies are visually connected to
> it, and they are used to construct the git references that Zuul
> uses when testing. When looking at this graph on the status page,
> you will note that the dependencies show up as grey dots, while the
> actual change tested shows up as red or green. This is to indicate
> that the grey changes are only there to establish dependencies.
> Even if one of the dependencies is also being tested, it will show
> up as a grey dot when used as a dependency, but separately and
> additionally will appear as its own red or green dot for its test.
>
> (If you don't see grey dots on the status page, reload the page to
> get the latest version.)
>
> Multiple Changes
>
> A Gerrit change ID may refer to multiple changes (on multiple
> branches of the same project, or even multiple projects). In these
> cases, Zuul will treat all of the changes with that change ID as
> dependencies. So if you say that a tempest change Depends-On a
> change ID that has changes in nova master and nova stable/juno,
> then when testing the tempest change, both nova changes will be
> applied, and when deciding whether the tempest change can merge,
> both changes must merge ahead of it.
>
> A change may depend on more than one Gerrit change ID as well. So
> it is possible for a change in tempest to depend on a change in
> devstack and a change in nova. Simply add more "Depends-On:" lines
> to the footer.
>
> Cycles ==
>
> If a cycle is created by use of CRD, Zuul will abort its work very
> early. There will be no message in Gerrit and no changes that are
> part of the cycle will be enqueued into any pipeline. This is to
> protect Zuul from infinite loops. I hope t
[openstack-dev] Cross-Repo Dependencies in Zuul
Hi,
We have added support for cross-repo dependencies (CRD) in Zuul. The
important bits:
* To use them, include "Depends-On: " in the footer of
your commit message. Use the full Change-ID ('I' + 40 characters).
* These are one-way dependencies only -- do not create a cycle.
* This is what all the grey dots and lines are in the check pipeline.
Cross-Repo Dependencies Explained
=
There are two behaviors that might go by the name "cross-repo
dependencies". We call them one-way and multi-way.
Multi-way CRD allow for bidirectional links between changes. For
instance, A depends on B and B depends on A. The theory there is that
both would be tested together and merged as a unit. This is _not_ what
we have implemented in Zuul. Discussions over the past two years have
revealed that this type of behavior could cause problems for continuous
deploments as it means that two components must be upgraded
simultaneously. Not supporting this behavior is a choice we have made.
One-way CRD behaves like a directed acyclic graph (DAG), like git
itself, to indicate a one-way dependency relationship between changes in
different git repos. Change A may depend on B, but B may not depend on
A. This is what we have implemented in Zuul.
Gate Pipeline
=
When Zuul sees CRD changes, it serializes them in the usual manner when
enqueuing them into a pipeline. This means that if change A depends on
B, then when they are added to the gate pipeline, B will appear first
and A will follow. If tests for B fail, both B and A will be removed
from the pipeline, and it will not be possible for A to merge until B
does.
Note that if changes with CRD do not share a change queue (such as the
"integrated gate" then Zuul is unable to enqueue them together, and the
first will be required to merge before the second is enqueued.
Check Pipeline
==
When changes are enqueued into the check pipeline, all of the related
dependencies (both normal git-dependencies that come from parent commits
as well as CRD changes) appear in a dependency graph, as in gate. This
means that even in the check pipeline, your change will be tested with
its dependency. So changes that were previously unable to be fully
tested until a related change landed in a different repo may now be
tested together from the start.
All of the changes are still independent (so you will note that the
whole pipeline does not share a graph as in gate), but for each change
tested, all of its dependencies are visually connected to it, and they
are used to construct the git references that Zuul uses when testing.
When looking at this graph on the status page, you will note that the
dependencies show up as grey dots, while the actual change tested shows
up as red or green. This is to indicate that the grey changes are only
there to establish dependencies. Even if one of the dependencies is
also being tested, it will show up as a grey dot when used as a
dependency, but separately and additionally will appear as its own red
or green dot for its test.
(If you don't see grey dots on the status page, reload the page to get
the latest version.)
Multiple Changes
A Gerrit change ID may refer to multiple changes (on multiple branches
of the same project, or even multiple projects). In these cases, Zuul
will treat all of the changes with that change ID as dependencies. So
if you say that a tempest change Depends-On a change ID that has changes
in nova master and nova stable/juno, then when testing the tempest
change, both nova changes will be applied, and when deciding whether the
tempest change can merge, both changes must merge ahead of it.
A change may depend on more than one Gerrit change ID as well. So it is
possible for a change in tempest to depend on a change in devstack and a
change in nova. Simply add more "Depends-On:" lines to the footer.
Cycles
==
If a cycle is created by use of CRD, Zuul will abort its work very
early. There will be no message in Gerrit and no changes that are part
of the cycle will be enqueued into any pipeline. This is to protect
Zuul from infinite loops. I hope that we can improve this to at least
leave a message in Gerrit in the future. But in the meantime, please be
cognizant of this and do not create dependency cycles with Depends-On
lines.
Examples
The following two infra changes have been tested together because of the
Depends-On: line in the commit message of the first:
https://review.openstack.org/#/c/152508/
https://review.openstack.org/#/c/152504/
In fact, you can see earlier test results failing until it was rechecked
after CRD went into production (around 2015-02-10 15:20 UTC).
This devstack change depended on a grenade change:
https://review.openstack.org/#/c/154575/
https://review.openstack.org/#/c/153702/
And with CRD, was able to be tested in check together as well as being
enqueued into the gate at the same time as its depen
