Re: [openstack-dev] [third-party] - rebasing patches for CI
I am experiencing the exact same issue. Several reviews did not rebase recently, which make some of my tests fail. What should be the behaviour from the CI point of view ? 1) -1 and request a rebase 2) 0 and request a rebase 3) Ci to do the cherry pick but like Kevin was stating, it need some custom code that I want to avoid Thanks, Franck Franck On Thu, Jul 24, 2014 at 4:15 PM, Kevin Benton blak...@gmail.com wrote: Cherry-picking onto the target branch requires an extra step and custom code that I wanted to avoid. Right now I can just pass the gerrit ref into devstack's local.conf as the branch and everything works. If there was a way to get that Zuul ref, I could just use that instead and no new code would be required. Is exposing that ref in a known format/location something the infra team might consider? Thanks On Tue, Jul 22, 2014 at 4:16 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-07-21 11:36:43 -0700 (-0700), Kevin Benton wrote: I see. So then back to my other question, is it possible to get access to the same branch that is being passed to the OpenStack CI devstack tests? For example, in the console output I can see it uses a ref like refs/zuul/ master/Z75ac747d605b4eb28d4add7fa5b99890.[1] Is that visible somewhere (other than the logs of course) could be used in a third-party system? Right now, no. It's information passed from Zuul to a Jenkins master via Gearman, but as far as I know is currently only discoverable within the logs and the job parameters displayed in Jenkins. There has been some discussion in the past of Zuul providing some more detailed information to third-party systems (perhaps the capability to add them as additional Gearman workers) but that has never been fully fleshed out. For the case of independent pipelines (which is all I would expect a third-party CI to have any interest in running for the purpose of testing a proposed change) it should be entirely sufficient to cherry-pick a patch/series from our Gerrit onto the target branch. Only _dependent_ pipelines currently make use of Zuul's capability to provide a common ref representing a set of different changes across multiple projects, since independent pipelines will only ever have an available ZUUL_REF on a single project (the same project for which the change is being proposed). -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
Cherry-picking onto the target branch requires an extra step and custom code that I wanted to avoid. Right now I can just pass the gerrit ref into devstack's local.conf as the branch and everything works. If there was a way to get that Zuul ref, I could just use that instead and no new code would be required. Is exposing that ref in a known format/location something the infra team might consider? Thanks On Tue, Jul 22, 2014 at 4:16 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-07-21 11:36:43 -0700 (-0700), Kevin Benton wrote: I see. So then back to my other question, is it possible to get access to the same branch that is being passed to the OpenStack CI devstack tests? For example, in the console output I can see it uses a ref like refs/zuul/ master/Z75ac747d605b4eb28d4add7fa5b99890.[1] Is that visible somewhere (other than the logs of course) could be used in a third-party system? Right now, no. It's information passed from Zuul to a Jenkins master via Gearman, but as far as I know is currently only discoverable within the logs and the job parameters displayed in Jenkins. There has been some discussion in the past of Zuul providing some more detailed information to third-party systems (perhaps the capability to add them as additional Gearman workers) but that has never been fully fleshed out. For the case of independent pipelines (which is all I would expect a third-party CI to have any interest in running for the purpose of testing a proposed change) it should be entirely sufficient to cherry-pick a patch/series from our Gerrit onto the target branch. Only _dependent_ pipelines currently make use of Zuul's capability to provide a common ref representing a set of different changes across multiple projects, since independent pipelines will only ever have an available ZUUL_REF on a single project (the same project for which the change is being proposed). -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
On 2014-07-21 11:36:43 -0700 (-0700), Kevin Benton wrote: I see. So then back to my other question, is it possible to get access to the same branch that is being passed to the OpenStack CI devstack tests? For example, in the console output I can see it uses a ref like refs/zuul/ master/Z75ac747d605b4eb28d4add7fa5b99890.[1] Is that visible somewhere (other than the logs of course) could be used in a third-party system? Right now, no. It's information passed from Zuul to a Jenkins master via Gearman, but as far as I know is currently only discoverable within the logs and the job parameters displayed in Jenkins. There has been some discussion in the past of Zuul providing some more detailed information to third-party systems (perhaps the capability to add them as additional Gearman workers) but that has never been fully fleshed out. For the case of independent pipelines (which is all I would expect a third-party CI to have any interest in running for the purpose of testing a proposed change) it should be entirely sufficient to cherry-pick a patch/series from our Gerrit onto the target branch. Only _dependent_ pipelines currently make use of Zuul's capability to provide a common ref representing a set of different changes across multiple projects, since independent pipelines will only ever have an available ZUUL_REF on a single project (the same project for which the change is being proposed). -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
I see. So then back to my other question, is it possible to get access to the same branch that is being passed to the OpenStack CI devstack tests? For example, in the console output I can see it uses a ref like refs/zuul/master/Z75ac747d605b4eb28d4add7fa5b99890.[1] Is that visible somewhere (other than the logs of course) could be used in a third-party system? Thanks 1. http://logs.openstack.org/64/83664/17/check/gate-neutron-python27/db29f20/console.html.gz On Sun, Jul 13, 2014 at 3:36 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-07-13 00:09:11 -0700 (-0700), Kevin Benton wrote: [...] Does Zuul only cherry-pick the top commit of the proposed patch instead of merging the proposed patch's branch into master (which would merge all dependent patchsets)? In an independent pipeline, Zuul tests the change as merged to the tip of the target branch along with any other changes that change depends on in Gerrit. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
3rd party systems do need to test the patch rebased against master since that's what OpenStack is testing. Otherwise, there are conditions where the patch can fail in its current position but not rebased against master so it will look like something is wrong with the 3rd party system. Similarly, the patch might pass without being rebased and then fail once merged into master and cause all subsequent patches to fail for the 3rd party system. Well, if you aren't using Zuul to handle the merging of dependent patchsets, I'm not entirely sure the ZUUL_ environment variables are going to help much. Can you elaborate on this some more? Does Zuul only cherry-pick the top commit of the proposed patch instead of merging the proposed patch's branch into master (which would merge all dependent patchsets)? On Sun, Jul 6, 2014 at 5:53 PM, Jay Pipes jaypi...@gmail.com wrote: On 07/03/2014 04:37 PM, Kevin Benton wrote: Are these zuul refs publicly accessible so that the third party CI systems could reference then to guarantee they are testing the same thing? Well, if you aren't using Zuul to handle the merging of dependent patchsets, I'm not entirely sure the ZUUL_ environment variables are going to help much. That said, if all you want is to test the specific branch/patch that is proposed in a Gerrit code review, then you just need to check out the commit that is referenced on the code review. For instance, if you go to this review and patchset: https://review.openstack.org/#/c/93860/2 the git branch and HEAD you would check out to test the code as it is in the code review would be: git fetch https://review.openstack.org/openstack/nova refs/changes/60/93860/2 git checkout FETCH_HEAD What Zuul does is provide support for managing *dependent patch queues*, allowing jobs to be aborted if a dependent patch has failed to merge or successfully run its test jobs. You don't need to test whether that patch will merge into master, since the upstream gate already does that. Hope that helps, -jay On Thu, Jul 3, 2014 at 11:31 AM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: On 07/03/2014 02:10 PM, Kevin Benton wrote: The reason I thought it changed was that this is the first cycle where I have encountered scenarios where my unit tests for the patch run fine locally, but then they fail when they are checked by Jenkins (caused by a change after the parent of my patch). I suppose I was just lucky before and never had anything merge after I proposed a patch that caused a conflict with mine. I suspect this is a problem then for many third-party CI systems because the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in devstack to point to the gerrit server will not work correctly since it will just test the patch without merging it. Where is this merging process handled in the OpenStack CI? Is that done in Zuul with the custom Zuul branch is passed to devstack? Yes. The zuul-merger daemon is responsible for managing this, and the devstack-gate project handles the checkout and setup of the git repos for all of the OpenStack projects. Best, -jay -- Kevin Benton On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley fu...@yuggoth.org mailto:fu...@yuggoth.org mailto:fu...@yuggoth.org mailto:fu...@yuggoth.org wrote: On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote: [...] As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. [...] I'm not sure what you think changed, but we've (upstream OpenStack CI) been testing proposed patches merged to their target branches for years... -- Jeremy Stanley _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.__openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__ openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/ openstack-dev -- Kevin Benton _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__ openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/ openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
On 2014-07-13 00:09:11 -0700 (-0700), Kevin Benton wrote: [...] Does Zuul only cherry-pick the top commit of the proposed patch instead of merging the proposed patch's branch into master (which would merge all dependent patchsets)? In an independent pipeline, Zuul tests the change as merged to the tip of the target branch along with any other changes that change depends on in Gerrit. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
On 07/03/2014 04:37 PM, Kevin Benton wrote: Are these zuul refs publicly accessible so that the third party CI systems could reference then to guarantee they are testing the same thing? Well, if you aren't using Zuul to handle the merging of dependent patchsets, I'm not entirely sure the ZUUL_ environment variables are going to help much. That said, if all you want is to test the specific branch/patch that is proposed in a Gerrit code review, then you just need to check out the commit that is referenced on the code review. For instance, if you go to this review and patchset: https://review.openstack.org/#/c/93860/2 the git branch and HEAD you would check out to test the code as it is in the code review would be: git fetch https://review.openstack.org/openstack/nova refs/changes/60/93860/2 git checkout FETCH_HEAD What Zuul does is provide support for managing *dependent patch queues*, allowing jobs to be aborted if a dependent patch has failed to merge or successfully run its test jobs. You don't need to test whether that patch will merge into master, since the upstream gate already does that. Hope that helps, -jay On Thu, Jul 3, 2014 at 11:31 AM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: On 07/03/2014 02:10 PM, Kevin Benton wrote: The reason I thought it changed was that this is the first cycle where I have encountered scenarios where my unit tests for the patch run fine locally, but then they fail when they are checked by Jenkins (caused by a change after the parent of my patch). I suppose I was just lucky before and never had anything merge after I proposed a patch that caused a conflict with mine. I suspect this is a problem then for many third-party CI systems because the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in devstack to point to the gerrit server will not work correctly since it will just test the patch without merging it. Where is this merging process handled in the OpenStack CI? Is that done in Zuul with the custom Zuul branch is passed to devstack? Yes. The zuul-merger daemon is responsible for managing this, and the devstack-gate project handles the checkout and setup of the git repos for all of the OpenStack projects. Best, -jay -- Kevin Benton On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley fu...@yuggoth.org mailto:fu...@yuggoth.org mailto:fu...@yuggoth.org mailto:fu...@yuggoth.org wrote: On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote: [...] As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. [...] I'm not sure what you think changed, but we've (upstream OpenStack CI) been testing proposed patches merged to their target branches for years... -- Jeremy Stanley _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.__openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
The reason I thought it changed was that this is the first cycle where I have encountered scenarios where my unit tests for the patch run fine locally, but then they fail when they are checked by Jenkins (caused by a change after the parent of my patch). I suppose I was just lucky before and never had anything merge after I proposed a patch that caused a conflict with mine. I suspect this is a problem then for many third-party CI systems because the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in devstack to point to the gerrit server will not work correctly since it will just test the patch without merging it. Where is this merging process handled in the OpenStack CI? Is that done in Zuul with the custom Zuul branch is passed to devstack? -- Kevin Benton On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote: [...] As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. [...] I'm not sure what you think changed, but we've (upstream OpenStack CI) been testing proposed patches merged to their target branches for years... -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
On 07/03/2014 02:10 PM, Kevin Benton wrote: The reason I thought it changed was that this is the first cycle where I have encountered scenarios where my unit tests for the patch run fine locally, but then they fail when they are checked by Jenkins (caused by a change after the parent of my patch). I suppose I was just lucky before and never had anything merge after I proposed a patch that caused a conflict with mine. I suspect this is a problem then for many third-party CI systems because the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in devstack to point to the gerrit server will not work correctly since it will just test the patch without merging it. Where is this merging process handled in the OpenStack CI? Is that done in Zuul with the custom Zuul branch is passed to devstack? Yes. The zuul-merger daemon is responsible for managing this, and the devstack-gate project handles the checkout and setup of the git repos for all of the OpenStack projects. Best, -jay -- Kevin Benton On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley fu...@yuggoth.org mailto:fu...@yuggoth.org wrote: On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote: [...] As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. [...] I'm not sure what you think changed, but we've (upstream OpenStack CI) been testing proposed patches merged to their target branches for years... -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
Are these zuul refs publicly accessible so that the third party CI systems could reference then to guarantee they are testing the same thing? On Thu, Jul 3, 2014 at 11:31 AM, Jay Pipes jaypi...@gmail.com wrote: On 07/03/2014 02:10 PM, Kevin Benton wrote: The reason I thought it changed was that this is the first cycle where I have encountered scenarios where my unit tests for the patch run fine locally, but then they fail when they are checked by Jenkins (caused by a change after the parent of my patch). I suppose I was just lucky before and never had anything merge after I proposed a patch that caused a conflict with mine. I suspect this is a problem then for many third-party CI systems because the simple approach of setting [PROJECT]_REPO and [PROJECT]_BRANCH in devstack to point to the gerrit server will not work correctly since it will just test the patch without merging it. Where is this merging process handled in the OpenStack CI? Is that done in Zuul with the custom Zuul branch is passed to devstack? Yes. The zuul-merger daemon is responsible for managing this, and the devstack-gate project handles the checkout and setup of the git repos for all of the OpenStack projects. Best, -jay -- Kevin Benton On Tue, Jul 1, 2014 at 4:00 PM, Jeremy Stanley fu...@yuggoth.org mailto:fu...@yuggoth.org wrote: On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote: [...] As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. [...] I'm not sure what you think changed, but we've (upstream OpenStack CI) been testing proposed patches merged to their target branches for years... -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [third-party] - rebasing patches for CI
Hello, What is the expected behavior of 3rd-party CI systems with regard to checking out a patch. Should it be tested 'as-is' or should it be merged into the proposed branch first and then tested? As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. Matching its behavior makes the most sense, especially since the 3rd party CI isn't running in the gate so it's the closest alternative. Is this what other CIs are doing? I didn't see anything about it in the Neutron testing wiki[1] so I figured I would bring this to the mailing list to get feedback. 1. https://wiki.openstack.org/wiki/NeutronThirdPartyTesting Cheers -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote: [...] As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. [...] I'm not sure what you think changed, but we've (upstream OpenStack CI) been testing proposed patches merged to their target branches for years... -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [third-party] - rebasing patches for CI
It could be that it this behavior has merely become more noticeable since Jenkins is now reverifying patch sets when new comments show up and it sees that the patch set hasn't been verified recently. Carl On Tue, Jul 1, 2014 at 5:00 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-07-01 10:05:45 -0700 (-0700), Kevin Benton wrote: [...] As I understand it, this behavior for the main OpenStack CI check queue changed to the latter some time over the past few months. [...] I'm not sure what you think changed, but we've (upstream OpenStack CI) been testing proposed patches merged to their target branches for years... -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
