Re: [openstack-dev] [third-party] - rebasing patches for CI

[Franck Yelles]

I am experiencing the exact same issue. Several reviews did not rebase
recently, which make some of my tests fail.
What should be the behaviour from the CI point of view ?
1) -1 and request a rebase
2) 0 and request a rebase
3) Ci to do the cherry pick but like Kevin was stating, it need some
custom code that I want to avoid

Thanks,
Franck
Franck


On Thu, Jul 24, 2014 at 4:15 PM, Kevin Benton  wrote:
> Cherry-picking onto the target branch requires an extra step and custom code
> that I wanted to avoid.
> Right now I can just pass the gerrit ref into devstack's local.conf as the
> branch and everything works.
> If there was a way to get that Zuul ref, I could just use that instead and
> no new code would be required.
>
> Is exposing that ref in a known format/location something the infra team
> might consider?
>
> Thanks
>
>
> On Tue, Jul 22, 2014 at 4:16 PM, Jeremy Stanley  wrote:
>>
>> On 2014-07-21 11:36:43 -0700 (-0700), Kevin Benton wrote:
>> > I see. So then back to my other question, is it possible to get
>> > access to the same branch that is being passed to the OpenStack CI
>> > devstack tests?
>> >
>> > For example, in the console output I can see it uses a ref
>> > like refs/zuul/ master/Z75ac747d605b4eb28d4add7fa5b99890.[1] Is
>> > that visible somewhere (other than the logs of course) could be
>> > used in a third-party system?
>>
>> Right now, no. It's information passed from Zuul to a Jenkins master
>> via Gearman, but as far as I know is currently only discoverable
>> within the logs and the job parameters displayed in Jenkins. There
>> has been some discussion in the past of Zuul providing some more
>> detailed information to third-party systems (perhaps the capability
>> to add them as additional Gearman workers) but that has never been
>> fully fleshed out.
>>
>> For the case of independent pipelines (which is all I would expect a
>> third-party CI to have any interest in running for the purpose of
>> testing a proposed change) it should be entirely sufficient to
>> cherry-pick a patch/series from our Gerrit onto the target branch.
>> Only _dependent_ pipelines currently make use of Zuul's capability
>> to provide a common ref representing a set of different changes
>> across multiple projects, since independent pipelines will only ever
>> have an available ZUUL_REF on a single project (the same project for
>> which the change is being proposed).
>> --
>> Jeremy Stanley
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Kevin Benton
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Kevin Benton]

Cherry-picking onto the target branch requires an extra step and custom
code that I wanted to avoid.
Right now I can just pass the gerrit ref into devstack's local.conf as the
branch and everything works.
 If there was a way to get that Zuul ref, I could just use that instead and
no new code would be required.

Is exposing that ref in a known format/location something the infra team
might consider?

Thanks


On Tue, Jul 22, 2014 at 4:16 PM, Jeremy Stanley  wrote:

> On 2014-07-21 11:36:43 -0700 (-0700), Kevin Benton wrote:
> > I see. So then back to my other question, is it possible to get
> > access to the same branch that is being passed to the OpenStack CI
> > devstack tests?
> >
> > For example, in the console output I can see it uses a ref
> > like refs/zuul/ master/Z75ac747d605b4eb28d4add7fa5b99890.[1] Is
> > that visible somewhere (other than the logs of course) could be
> > used in a third-party system?
>
> Right now, no. It's information passed from Zuul to a Jenkins master
> via Gearman, but as far as I know is currently only discoverable
> within the logs and the job parameters displayed in Jenkins. There
> has been some discussion in the past of Zuul providing some more
> detailed information to third-party systems (perhaps the capability
> to add them as additional Gearman workers) but that has never been
> fully fleshed out.
>
> For the case of independent pipelines (which is all I would expect a
> third-party CI to have any interest in running for the purpose of
> testing a proposed change) it should be entirely sufficient to
> cherry-pick a patch/series from our Gerrit onto the target branch.
> Only _dependent_ pipelines currently make use of Zuul's capability
> to provide a common ref representing a set of different changes
> across multiple projects, since independent pipelines will only ever
> have an available ZUUL_REF on a single project (the same project for
> which the change is being proposed).
> --
> Jeremy Stanley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Jeremy Stanley]

On 2014-07-21 11:36:43 -0700 (-0700), Kevin Benton wrote:
> I see. So then back to my other question, is it possible to get
> access to the same branch that is being passed to the OpenStack CI
> devstack tests?
> 
> For example, in the console output I can see it uses a ref
> like refs/zuul/ master/Z75ac747d605b4eb28d4add7fa5b99890.[1] Is
> that visible somewhere (other than the logs of course) could be
> used in a third-party system?

Right now, no. It's information passed from Zuul to a Jenkins master
via Gearman, but as far as I know is currently only discoverable
within the logs and the job parameters displayed in Jenkins. There
has been some discussion in the past of Zuul providing some more
detailed information to third-party systems (perhaps the capability
to add them as additional Gearman workers) but that has never been
fully fleshed out.

For the case of independent pipelines (which is all I would expect a
third-party CI to have any interest in running for the purpose of
testing a proposed change) it should be entirely sufficient to
cherry-pick a patch/series from our Gerrit onto the target branch.
Only _dependent_ pipelines currently make use of Zuul's capability
to provide a common ref representing a set of different changes
across multiple projects, since independent pipelines will only ever
have an available ZUUL_REF on a single project (the same project for
which the change is being proposed).
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Kevin Benton]

I see. So then back to my other question, is it possible to get access to
the same branch that is being passed to the OpenStack CI devstack tests?

For example, in the console output I can see it uses a ref
like refs/zuul/master/Z75ac747d605b4eb28d4add7fa5b99890.[1]
Is that visible somewhere (other than the logs of course) could be used in
a third-party system?

Thanks

1.
http://logs.openstack.org/64/83664/17/check/gate-neutron-python27/db29f20/console.html.gz


On Sun, Jul 13, 2014 at 3:36 PM, Jeremy Stanley  wrote:

> On 2014-07-13 00:09:11 -0700 (-0700), Kevin Benton wrote:
> [...]
> > Does Zuul only cherry-pick the top commit of the proposed patch
> > instead of merging the proposed patch's branch into master (which
> > would merge all dependent patchsets)?
>
> In an independent pipeline, Zuul tests the change as merged to the
> tip of the target branch along with any other changes that change
> depends on in Gerrit.
> --
> Jeremy Stanley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Jeremy Stanley]

On 2014-07-13 00:09:11 -0700 (-0700), Kevin Benton wrote:
[...]
> Does Zuul only cherry-pick the top commit of the proposed patch
> instead of merging the proposed patch's branch into master (which
> would merge all dependent patchsets)?

In an independent pipeline, Zuul tests the change as merged to the
tip of the target branch along with any other changes that change
depends on in Gerrit.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Kevin Benton]

3rd party systems do need to test the patch rebased against master since
that's what OpenStack is testing. Otherwise, there are conditions where the
patch can fail in its current position but not rebased against master so it
will look like something is wrong with the 3rd party system. Similarly, the
patch might pass without being rebased and then fail once merged into
master and cause all subsequent patches to fail for the 3rd party system.

>Well, if you aren't using Zuul to handle the merging of dependent
patchsets, I'm not entirely sure the ZUUL_ environment variables are going
to help much.

Can you elaborate on this some more? Does Zuul only cherry-pick the top
commit of the proposed patch instead of merging the proposed patch's branch
into master (which would merge all dependent patchsets)?


On Sun, Jul 6, 2014 at 5:53 PM, Jay Pipes  wrote:

> On 07/03/2014 04:37 PM, Kevin Benton wrote:
>
>> Are these zuul refs publicly accessible so that the third party CI
>> systems could reference then to guarantee they are testing the same thing?
>>
>
> Well, if you aren't using Zuul to handle the merging of dependent
> patchsets, I'm not entirely sure the ZUUL_ environment variables are going
> to help much.
>
> That said, if all you want is to test the specific branch/patch that is
> proposed in a Gerrit code review, then you just need to check out the
> commit that is referenced on the code review. For instance, if you go to
> this review and patchset:
>
> https://review.openstack.org/#/c/93860/2
>
> the git branch and HEAD you would check out to test the code as it is in
> the code review would be:
>
> git fetch https://review.openstack.org/openstack/nova
> refs/changes/60/93860/2 && git checkout FETCH_HEAD
>
> What Zuul does is provide support for managing *dependent patch queues*,
> allowing jobs to be aborted if a dependent patch has failed to merge or
> successfully run its test jobs.
>
> You don't need to test whether that patch will merge into master, since
> the upstream gate already does that.
>
> Hope that helps,
> -jay
>
>  On Thu, Jul 3, 2014 at 11:31 AM, Jay Pipes > > wrote:
>>
>> On 07/03/2014 02:10 PM, Kevin Benton wrote:
>>
>> The reason I thought it changed was that this is the first cycle
>> where I
>> have encountered scenarios where my unit tests for the patch run
>> fine
>> locally, but then they fail when they are checked by Jenkins
>> (caused by
>> a change after the parent of my patch). I suppose I was just lucky
>> before and never had anything merge after I proposed a patch
>> that caused
>> a conflict with mine.
>>
>> I suspect this is a problem then for many third-party CI systems
>> because
>> the simple approach of setting [PROJECT]_REPO and
>> [PROJECT]_BRANCH in
>> devstack to point to the gerrit server will not work correctly
>> since it
>> will just test the patch without merging it.
>>
>> Where is this merging process handled in the OpenStack CI? Is
>> that done
>> in Zuul with the custom Zuul branch is passed to devstack?
>>
>>
>> Yes. The zuul-merger daemon is responsible for managing this, and
>> the devstack-gate project handles the checkout and setup of the git
>> repos for all of the OpenStack projects.
>>
>> Best,
>> -jay
>>
>> --
>> Kevin Benton
>>
>>
>> On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley
>> mailto:fu...@yuggoth.org>
>> >> wrote:
>>
>>  On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote:
>>  [...]
>>   > As I understand it, this behavior for the main OpenStack
>> CI check
>>   > queue changed to the latter some time over the past few
>> months.
>>  [...]
>>
>>  I'm not sure what you think changed, but we've (upstream
>> OpenStack
>>  CI) been testing proposed patches merged to their target
>> branches
>>  for years...
>>  --
>>  Jeremy Stanley
>>
>>  _
>>  OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.__org
>> 
>>  > >
>>
>> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__
>> openstack-dev
>>
>> > openstack-dev>
>>
>>
>>
>>
>> --
>> Kevin Benton
>>
>>
>> _
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.__org
>> 
>> http:

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Jay Pipes]

On 07/03/2014 04:37 PM, Kevin Benton wrote:

Are these zuul refs publicly accessible so that the third party CI
systems could reference then to guarantee they are testing the same thing?


Well, if you aren't using Zuul to handle the merging of dependent 
patchsets, I'm not entirely sure the ZUUL_ environment variables are 
going to help much.


That said, if all you want is to test the specific branch/patch that is 
proposed in a Gerrit code review, then you just need to check out the 
commit that is referenced on the code review. For instance, if you go to 
this review and patchset:


https://review.openstack.org/#/c/93860/2

the git branch and HEAD you would check out to test the code as it is in 
the code review would be:


git fetch https://review.openstack.org/openstack/nova 
refs/changes/60/93860/2 && git checkout FETCH_HEAD


What Zuul does is provide support for managing *dependent patch queues*, 
allowing jobs to be aborted if a dependent patch has failed to merge or 
successfully run its test jobs.


You don't need to test whether that patch will merge into master, since 
the upstream gate already does that.


Hope that helps,
-jay


On Thu, Jul 3, 2014 at 11:31 AM, Jay Pipes mailto:jaypi...@gmail.com>> wrote:

On 07/03/2014 02:10 PM, Kevin Benton wrote:

The reason I thought it changed was that this is the first cycle
where I
have encountered scenarios where my unit tests for the patch run
fine
locally, but then they fail when they are checked by Jenkins
(caused by
a change after the parent of my patch). I suppose I was just lucky
before and never had anything merge after I proposed a patch
that caused
a conflict with mine.

I suspect this is a problem then for many third-party CI systems
because
the simple approach of setting [PROJECT]_REPO and
[PROJECT]_BRANCH in
devstack to point to the gerrit server will not work correctly
since it
will just test the patch without merging it.

Where is this merging process handled in the OpenStack CI? Is
that done
in Zuul with the custom Zuul branch is passed to devstack?


Yes. The zuul-merger daemon is responsible for managing this, and
the devstack-gate project handles the checkout and setup of the git
repos for all of the OpenStack projects.

Best,
-jay

--
Kevin Benton


On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley
mailto:fu...@yuggoth.org>
>> wrote:

 On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote:
 [...]
  > As I understand it, this behavior for the main OpenStack
CI check
  > queue changed to the latter some time over the past few
months.
 [...]

 I'm not sure what you think changed, but we've (upstream
OpenStack
 CI) been testing proposed patches merged to their target
branches
 for years...
 --
 Jeremy Stanley

 _
 OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

 >

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev





--
Kevin Benton


_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev



_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 





--
Kevin Benton


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Kevin Benton]

Are these zuul refs publicly accessible so that the third party CI systems
could reference then to guarantee they are testing the same thing?


On Thu, Jul 3, 2014 at 11:31 AM, Jay Pipes  wrote:

> On 07/03/2014 02:10 PM, Kevin Benton wrote:
>
>> The reason I thought it changed was that this is the first cycle where I
>> have encountered scenarios where my unit tests for the patch run fine
>> locally, but then they fail when they are checked by Jenkins (caused by
>> a change after the parent of my patch). I suppose I was just lucky
>> before and never had anything merge after I proposed a patch that caused
>> a conflict with mine.
>>
>> I suspect this is a problem then for many third-party CI systems because
>> the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in
>> devstack to point to the gerrit server will not work correctly since it
>> will just test the patch without merging it.
>>
>> Where is this merging process handled in the OpenStack CI? Is that done
>> in Zuul with the custom Zuul branch is passed to devstack?
>>
>
> Yes. The zuul-merger daemon is responsible for managing this, and the
> devstack-gate project handles the checkout and setup of the git repos for
> all of the OpenStack projects.
>
> Best,
> -jay
>
>  --
>> Kevin Benton
>>
>>
>> On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley > > wrote:
>>
>> On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote:
>> [...]
>>  > As I understand it, this behavior for the main OpenStack CI check
>>  > queue changed to the latter some time over the past few months.
>> [...]
>>
>> I'm not sure what you think changed, but we've (upstream OpenStack
>> CI) been testing proposed patches merged to their target branches
>> for years...
>> --
>> Jeremy Stanley
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> 
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> --
>> Kevin Benton
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Jay Pipes]

On 07/03/2014 02:10 PM, Kevin Benton wrote:

The reason I thought it changed was that this is the first cycle where I
have encountered scenarios where my unit tests for the patch run fine
locally, but then they fail when they are checked by Jenkins (caused by
a change after the parent of my patch). I suppose I was just lucky
before and never had anything merge after I proposed a patch that caused
a conflict with mine.

I suspect this is a problem then for many third-party CI systems because
the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in
devstack to point to the gerrit server will not work correctly since it
will just test the patch without merging it.

Where is this merging process handled in the OpenStack CI? Is that done
in Zuul with the custom Zuul branch is passed to devstack?


Yes. The zuul-merger daemon is responsible for managing this, and the 
devstack-gate project handles the checkout and setup of the git repos 
for all of the OpenStack projects.


Best,
-jay


--
Kevin Benton


On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley mailto:fu...@yuggoth.org>> wrote:

On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote:
[...]
 > As I understand it, this behavior for the main OpenStack CI check
 > queue changed to the latter some time over the past few months.
[...]

I'm not sure what you think changed, but we've (upstream OpenStack
CI) been testing proposed patches merged to their target branches
for years...
--
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Kevin Benton


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Kevin Benton]

The reason I thought it changed was that this is the first cycle where I
have encountered scenarios where my unit tests for the patch run fine
locally, but then they fail when they are checked by Jenkins (caused by a
change after the parent of my patch). I suppose I was just lucky before and
never had anything merge after I proposed a patch that caused a conflict
with mine.

I suspect this is a problem then for many third-party CI systems because
the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in
devstack to point to the gerrit server will not work correctly since it
will just test the patch without merging it.

Where is this merging process handled in the OpenStack CI? Is that done in
Zuul with the custom Zuul branch is passed to devstack?

--
Kevin Benton


On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley  wrote:

> On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote:
> [...]
> > As I understand it, this behavior for the main OpenStack CI check
> > queue changed to the latter some time over the past few months.
> [...]
>
> I'm not sure what you think changed, but we've (upstream OpenStack
> CI) been testing proposed patches merged to their target branches
> for years...
> --
> Jeremy Stanley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Carl Baldwin]

It could be that it this behavior has merely become more noticeable
since Jenkins is now reverifying patch sets when new comments show up
and it sees that the patch set hasn't been verified recently.

Carl

On Tue, Jul 1, 2014 at 5:00 PM, Jeremy Stanley  wrote:
> On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote:
> [...]
>> As I understand it, this behavior for the main OpenStack CI check
>> queue changed to the latter some time over the past few months.
> [...]
>
> I'm not sure what you think changed, but we've (upstream OpenStack
> CI) been testing proposed patches merged to their target branches
> for years...
> --
> Jeremy Stanley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [third-party] - rebasing patches for CI

[Jeremy Stanley]

On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote:
[...]
> As I understand it, this behavior for the main OpenStack CI check
> queue changed to the latter some time over the past few months.
[...]

I'm not sure what you think changed, but we've (upstream OpenStack
CI) been testing proposed patches merged to their target branches
for years...
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev