Re: [openstack-dev] [Nova] [Ironic] [Infra] Making Ironic vote as a third-party Nova driver
Howdy, Here is my first pass at allowing different voters for different pipelines. https://review.openstack.org/#/c/97391/2 Cheers, Josh Rackspace Australia On 5/30/14 3:30 AM, Devananda van der Veen wrote: On Wed, May 28, 2014 at 10:54 PM, Joshua Hesketh mailto:joshua.hesk...@rackspace.com>> wrote: On 5/29/14 8:52 AM, James E. Blair wrote: Devananda van der Veen mailto:devananda@gmail.com>> writes: Hi all! This is a follow-up to several summit discussions on how-do-we-deprecate-baremetal, a summary of the plan forward, a call to raise awareness of the project's status, and hopefully gain some interest from folks on nova-core to help with spec and code reviews. The nova.virt.ironic driver lives in Ironic's git tree today [1]. We're cleaning it up and submitting it to Nova again this cycle. I've posted specs [2] outlining the design and planned upgrade process. Earlier today, we enabled voting in Ironic's check and gate queues for the tempest-dsvm-virtual-ironic job. This runs a tempest scenario test [3] against devstack, exercising Nova with the Ironic driver to PXE boot a virtual machine. It has been running for a few months on Ironic, and has been stable for more than a month. However, because Ironic is not integrated, we also can't vote in check/gate queues on integrated projects (like Nova). We can - and do - report the test result in a non-voting way, though that's easy to miss, since it looks like every other non-voting test. At the summit [4], it was suggested that we make this job report as though it were a third-party CI test for a Nova driver. This would be removed at the time that Ironic graduates and the job is allowed to vote in the gate. Until that time, I'm happy to have the nova.virt.ironic driver reporting as a third-party driver (even though it's not) simply to help raise awareness (third-party CI jobs are watched more closely than non-voting jobs) and decrease the likelihood that Nova developers will inadvertently break Ironic's gate. Given that there's a concrete plan forward, why am I sending this email to all three teams? A few reasons: - document the plan that we discussed - many people from infra and nova were not present during the discussion and may not be aware of the details - I may have gotten something wrong (it was a long week) - and mostly because I don't technically know how to make an upstream job report as though it's a third-party job, and am hoping someone wants to volunteer to help figure that out I think it's a reasonable plan. To elaborate a bit, I think we identified three categories of jobs that we run: a) jobs that are voting b) jobs that are non-voting because they are advisory c) jobs that are non-voting for policy reasons but we feel fairly strongly about There's a pretty subtle distinction between b and c. Ideally, there shouldn't be any. We've tried to minimize the number of non-voting jobs to make sure that people don't ignore them. Nonetheless, it seems that a large enough number of people still do that non-voting jobs are considered ineffective in Nova. I think it's worth noting the potential danger of de-emphasizing the actual results. It may make other non-voting jobs even less effective than they already are. The intent is to make the jobs described by (c) into voting jobs, but in a way that they can still be overridden if need be. The aim is to help new (eg, incubated) projects join the integrated gate in a way that lets them prove they are sufficiently mature to do so without impacting the currently integrated projects. I believe we're currently thinking that point is after their integration approval. If we are comfortable with incubated projects being able to block the integrated gate earlier, we could simply make the non-voting jobs voting instead. Back to the proposal at hand. I think we should call the kinds of jobs described in (c) as "non-binding". The best way to do that is to register a second user with Gerrit for Zuul to use, and have
Re: [openstack-dev] [Nova] [Ironic] [Infra] Making Ironic vote as a third-party Nova driver
On Wed, May 28, 2014 at 10:54 PM, Joshua Hesketh < joshua.hesk...@rackspace.com> wrote: > On 5/29/14 8:52 AM, James E. Blair wrote: > >> Devananda van der Veen writes: >> >> Hi all! >>> >>> This is a follow-up to several summit discussions on >>> how-do-we-deprecate-baremetal, a summary of the plan forward, a call to >>> raise awareness of the project's status, and hopefully gain some interest >>> from folks on nova-core to help with spec and code reviews. >>> >>> The nova.virt.ironic driver lives in Ironic's git tree today [1]. We're >>> cleaning it up and submitting it to Nova again this cycle. I've posted >>> specs [2] outlining the design and planned upgrade process. Earlier >>> today, >>> we enabled voting in Ironic's check and gate queues for the >>> tempest-dsvm-virtual-ironic job. This runs a tempest scenario test [3] >>> against devstack, exercising Nova with the Ironic driver to PXE boot a >>> virtual machine. It has been running for a few months on Ironic, and has >>> been stable for more than a month. However, because Ironic is not >>> integrated, we also can't vote in check/gate queues on integrated >>> projects >>> (like Nova). We can - and do - report the test result in a non-voting >>> way, >>> though that's easy to miss, since it looks like every other non-voting >>> test. >>> >>> At the summit [4], it was suggested that we make this job report as >>> though >>> it were a third-party CI test for a Nova driver. This would be removed at >>> the time that Ironic graduates and the job is allowed to vote in the >>> gate. >>> Until that time, I'm happy to have the nova.virt.ironic driver reporting >>> as >>> a third-party driver (even though it's not) simply to help raise >>> awareness >>> (third-party CI jobs are watched more closely than non-voting jobs) and >>> decrease the likelihood that Nova developers will inadvertently break >>> Ironic's gate. >>> >>> Given that there's a concrete plan forward, why am I sending this email >>> to >>> all three teams? A few reasons: >>> - document the plan that we discussed >>> - many people from infra and nova were not present during the discussion >>> and may not be aware of the details >>> - I may have gotten something wrong (it was a long week) >>> - and mostly because I don't technically know how to make an upstream job >>> report as though it's a third-party job, and am hoping someone wants to >>> volunteer to help figure that out >>> >> I think it's a reasonable plan. To elaborate a bit, I think we >> identified three categories of jobs that we run: >> >> a) jobs that are voting >> b) jobs that are non-voting because they are advisory >> c) jobs that are non-voting for policy reasons but we feel fairly >> strongly about >> >> There's a pretty subtle distinction between b and c. Ideally, there >> shouldn't be any. We've tried to minimize the number of non-voting jobs >> to make sure that people don't ignore them. Nonetheless, it seems that >> a large enough number of people still do that non-voting jobs are >> considered ineffective in Nova. I think it's worth noting the potential >> danger of de-emphasizing the actual results. It may make other >> non-voting jobs even less effective than they already are. >> >> The intent is to make the jobs described by (c) into voting jobs, but in >> a way that they can still be overridden if need be. The aim is to help >> new (eg, incubated) projects join the integrated gate in a way that lets >> them prove they are sufficiently mature to do so without impacting the >> currently integrated projects. I believe we're currently thinking that >> point is after their integration approval. If we are comfortable with >> incubated projects being able to block the integrated gate earlier, we >> could simply make the non-voting jobs voting instead. >> >> Back to the proposal at hand. I think we should call the kinds of jobs >> described in (c) as "non-binding". >> >> The best way to do that is to register a second user with Gerrit for >> Zuul to use, and have it report non-binding jobs with a +/- 1 vote in >> the check queue that is separate from the normal "Jenkins" vote. In >> order to do that, we will have to modify Zuul to be able to support a >> second user, and associate that user with a pipeline. Then configure a >> new "non-binding" pipeline to use that user and run the desired jobs. >> >> Note that a similar problem of curation may occur with the non-binding >> jobs. If we run jobs for the incubated projects Foo and Bar, they will >> share a vote in Gerrit, and Nova developers will have to examine the >> results of -1 votes; if Bar consistently fails tests, it may need to be >> made non-voting or removed to avoid obscuring Foo's results. >> >> I expect the Zuul modification to take an experienced Zuul developer >> about 2-3 days to write, or an inexperienced one about a week. If no >> one else has started it by then, I will probably have some time around >> the middle of the cycle to hack on i
Re: [openstack-dev] [Nova] [Ironic] [Infra] Making Ironic vote as a third-party Nova driver
On 5/29/14 8:52 AM, James E. Blair wrote: Devananda van der Veen writes: Hi all! This is a follow-up to several summit discussions on how-do-we-deprecate-baremetal, a summary of the plan forward, a call to raise awareness of the project's status, and hopefully gain some interest from folks on nova-core to help with spec and code reviews. The nova.virt.ironic driver lives in Ironic's git tree today [1]. We're cleaning it up and submitting it to Nova again this cycle. I've posted specs [2] outlining the design and planned upgrade process. Earlier today, we enabled voting in Ironic's check and gate queues for the tempest-dsvm-virtual-ironic job. This runs a tempest scenario test [3] against devstack, exercising Nova with the Ironic driver to PXE boot a virtual machine. It has been running for a few months on Ironic, and has been stable for more than a month. However, because Ironic is not integrated, we also can't vote in check/gate queues on integrated projects (like Nova). We can - and do - report the test result in a non-voting way, though that's easy to miss, since it looks like every other non-voting test. At the summit [4], it was suggested that we make this job report as though it were a third-party CI test for a Nova driver. This would be removed at the time that Ironic graduates and the job is allowed to vote in the gate. Until that time, I'm happy to have the nova.virt.ironic driver reporting as a third-party driver (even though it's not) simply to help raise awareness (third-party CI jobs are watched more closely than non-voting jobs) and decrease the likelihood that Nova developers will inadvertently break Ironic's gate. Given that there's a concrete plan forward, why am I sending this email to all three teams? A few reasons: - document the plan that we discussed - many people from infra and nova were not present during the discussion and may not be aware of the details - I may have gotten something wrong (it was a long week) - and mostly because I don't technically know how to make an upstream job report as though it's a third-party job, and am hoping someone wants to volunteer to help figure that out I think it's a reasonable plan. To elaborate a bit, I think we identified three categories of jobs that we run: a) jobs that are voting b) jobs that are non-voting because they are advisory c) jobs that are non-voting for policy reasons but we feel fairly strongly about There's a pretty subtle distinction between b and c. Ideally, there shouldn't be any. We've tried to minimize the number of non-voting jobs to make sure that people don't ignore them. Nonetheless, it seems that a large enough number of people still do that non-voting jobs are considered ineffective in Nova. I think it's worth noting the potential danger of de-emphasizing the actual results. It may make other non-voting jobs even less effective than they already are. The intent is to make the jobs described by (c) into voting jobs, but in a way that they can still be overridden if need be. The aim is to help new (eg, incubated) projects join the integrated gate in a way that lets them prove they are sufficiently mature to do so without impacting the currently integrated projects. I believe we're currently thinking that point is after their integration approval. If we are comfortable with incubated projects being able to block the integrated gate earlier, we could simply make the non-voting jobs voting instead. Back to the proposal at hand. I think we should call the kinds of jobs described in (c) as "non-binding". The best way to do that is to register a second user with Gerrit for Zuul to use, and have it report non-binding jobs with a +/- 1 vote in the check queue that is separate from the normal "Jenkins" vote. In order to do that, we will have to modify Zuul to be able to support a second user, and associate that user with a pipeline. Then configure a new "non-binding" pipeline to use that user and run the desired jobs. Note that a similar problem of curation may occur with the non-binding jobs. If we run jobs for the incubated projects Foo and Bar, they will share a vote in Gerrit, and Nova developers will have to examine the results of -1 votes; if Bar consistently fails tests, it may need to be made non-voting or removed to avoid obscuring Foo's results. I expect the Zuul modification to take an experienced Zuul developer about 2-3 days to write, or an inexperienced one about a week. If no one else has started it by then, I will probably have some time around the middle of the cycle to hack on it. In the mean time, we may want to make sure that the number of non-voting jobs is at a minimum (and further reduce them if possible), and emphasize to reviewers the importance of checking posted results. I like this plan. I can make a start on implementing this :-) Cheers, Josh -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org htt
Re: [openstack-dev] [Nova] [Ironic] [Infra] Making Ironic vote as a third-party Nova driver
Devananda van der Veen writes: > Hi all! > > This is a follow-up to several summit discussions on > how-do-we-deprecate-baremetal, a summary of the plan forward, a call to > raise awareness of the project's status, and hopefully gain some interest > from folks on nova-core to help with spec and code reviews. > > The nova.virt.ironic driver lives in Ironic's git tree today [1]. We're > cleaning it up and submitting it to Nova again this cycle. I've posted > specs [2] outlining the design and planned upgrade process. Earlier today, > we enabled voting in Ironic's check and gate queues for the > tempest-dsvm-virtual-ironic job. This runs a tempest scenario test [3] > against devstack, exercising Nova with the Ironic driver to PXE boot a > virtual machine. It has been running for a few months on Ironic, and has > been stable for more than a month. However, because Ironic is not > integrated, we also can't vote in check/gate queues on integrated projects > (like Nova). We can - and do - report the test result in a non-voting way, > though that's easy to miss, since it looks like every other non-voting test. > > At the summit [4], it was suggested that we make this job report as though > it were a third-party CI test for a Nova driver. This would be removed at > the time that Ironic graduates and the job is allowed to vote in the gate. > Until that time, I'm happy to have the nova.virt.ironic driver reporting as > a third-party driver (even though it's not) simply to help raise awareness > (third-party CI jobs are watched more closely than non-voting jobs) and > decrease the likelihood that Nova developers will inadvertently break > Ironic's gate. > > Given that there's a concrete plan forward, why am I sending this email to > all three teams? A few reasons: > - document the plan that we discussed > - many people from infra and nova were not present during the discussion > and may not be aware of the details > - I may have gotten something wrong (it was a long week) > - and mostly because I don't technically know how to make an upstream job > report as though it's a third-party job, and am hoping someone wants to > volunteer to help figure that out I think it's a reasonable plan. To elaborate a bit, I think we identified three categories of jobs that we run: a) jobs that are voting b) jobs that are non-voting because they are advisory c) jobs that are non-voting for policy reasons but we feel fairly strongly about There's a pretty subtle distinction between b and c. Ideally, there shouldn't be any. We've tried to minimize the number of non-voting jobs to make sure that people don't ignore them. Nonetheless, it seems that a large enough number of people still do that non-voting jobs are considered ineffective in Nova. I think it's worth noting the potential danger of de-emphasizing the actual results. It may make other non-voting jobs even less effective than they already are. The intent is to make the jobs described by (c) into voting jobs, but in a way that they can still be overridden if need be. The aim is to help new (eg, incubated) projects join the integrated gate in a way that lets them prove they are sufficiently mature to do so without impacting the currently integrated projects. I believe we're currently thinking that point is after their integration approval. If we are comfortable with incubated projects being able to block the integrated gate earlier, we could simply make the non-voting jobs voting instead. Back to the proposal at hand. I think we should call the kinds of jobs described in (c) as "non-binding". The best way to do that is to register a second user with Gerrit for Zuul to use, and have it report non-binding jobs with a +/- 1 vote in the check queue that is separate from the normal "Jenkins" vote. In order to do that, we will have to modify Zuul to be able to support a second user, and associate that user with a pipeline. Then configure a new "non-binding" pipeline to use that user and run the desired jobs. Note that a similar problem of curation may occur with the non-binding jobs. If we run jobs for the incubated projects Foo and Bar, they will share a vote in Gerrit, and Nova developers will have to examine the results of -1 votes; if Bar consistently fails tests, it may need to be made non-voting or removed to avoid obscuring Foo's results. I expect the Zuul modification to take an experienced Zuul developer about 2-3 days to write, or an inexperienced one about a week. If no one else has started it by then, I will probably have some time around the middle of the cycle to hack on it. In the mean time, we may want to make sure that the number of non-voting jobs is at a minimum (and further reduce them if possible), and emphasize to reviewers the importance of checking posted results. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/opensta
Re: [openstack-dev] [Nova] [Ironic] [Infra] Making Ironic vote as a third-party Nova driver
On Fri, May 23, 2014 at 7:38 PM, Devananda van der Veen < devananda@gmail.com> wrote: > Hi all! > > This is a follow-up to several summit discussions on > how-do-we-deprecate-baremetal, a summary of the plan forward, a call to > raise awareness of the project's status, and hopefully gain some interest > from folks on nova-core to help with spec and code reviews. > > The nova.virt.ironic driver lives in Ironic's git tree today [1]. We're > cleaning it up and submitting it to Nova again this cycle. I've posted > specs [2] outlining the design and planned upgrade process. Earlier today, > we enabled voting in Ironic's check and gate queues for the > tempest-dsvm-virtual-ironic job. This runs a tempest scenario test [3] > against devstack, exercising Nova with the Ironic driver to PXE boot a > virtual machine. It has been running for a few months on Ironic, and has > been stable for more than a month. However, because Ironic is not > integrated, we also can't vote in check/gate queues on integrated projects > (like Nova). We can - and do - report the test result in a non-voting way, > though that's easy to miss, since it looks like every other non-voting test. > > At the summit [4], it was suggested that we make this job report as though > it were a third-party CI test for a Nova driver. This would be removed at > the time that Ironic graduates and the job is allowed to vote in the gate. > Until that time, I'm happy to have the nova.virt.ironic driver reporting as > a third-party driver (even though it's not) simply to help raise awareness > (third-party CI jobs are watched more closely than non-voting jobs) and > decrease the likelihood that Nova developers will inadvertently break > Ironic's gate. > > Given that there's a concrete plan forward, why am I sending this email to > all three teams? A few reasons: > - document the plan that we discussed > - many people from infra and nova were not present during the discussion > and may not be aware of the details > - I may have gotten something wrong (it was a long week) > - and mostly because I don't technically know how to make an upstream job > report as though it's a third-party job, and am hoping someone wants to > volunteer to help figure that out > > Sounds like a great plan to me - this should help raise awareness in nova, although I cannot speak to what is needed to make ironic vote the same way a third party CI system does. > Regards, > Devananda > > > 1: https://github.com/openstack/ironic/tree/master/ironic/nova/virt/ironic > > 2: https://review.openstack.org/95024 and > https://review.openstack.org/95025 > > 3: > https://github.com/openstack/tempest/blob/master/tempest/scenario/test_baremetal_basic_ops.py > > 4: https://etherpad.openstack.org/p/juno-nova-deprecating-baremetal > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] [Ironic] [Infra] Making Ironic vote as a third-party Nova driver
Hi all! This is a follow-up to several summit discussions on how-do-we-deprecate-baremetal, a summary of the plan forward, a call to raise awareness of the project's status, and hopefully gain some interest from folks on nova-core to help with spec and code reviews. The nova.virt.ironic driver lives in Ironic's git tree today [1]. We're cleaning it up and submitting it to Nova again this cycle. I've posted specs [2] outlining the design and planned upgrade process. Earlier today, we enabled voting in Ironic's check and gate queues for the tempest-dsvm-virtual-ironic job. This runs a tempest scenario test [3] against devstack, exercising Nova with the Ironic driver to PXE boot a virtual machine. It has been running for a few months on Ironic, and has been stable for more than a month. However, because Ironic is not integrated, we also can't vote in check/gate queues on integrated projects (like Nova). We can - and do - report the test result in a non-voting way, though that's easy to miss, since it looks like every other non-voting test. At the summit [4], it was suggested that we make this job report as though it were a third-party CI test for a Nova driver. This would be removed at the time that Ironic graduates and the job is allowed to vote in the gate. Until that time, I'm happy to have the nova.virt.ironic driver reporting as a third-party driver (even though it's not) simply to help raise awareness (third-party CI jobs are watched more closely than non-voting jobs) and decrease the likelihood that Nova developers will inadvertently break Ironic's gate. Given that there's a concrete plan forward, why am I sending this email to all three teams? A few reasons: - document the plan that we discussed - many people from infra and nova were not present during the discussion and may not be aware of the details - I may have gotten something wrong (it was a long week) - and mostly because I don't technically know how to make an upstream job report as though it's a third-party job, and am hoping someone wants to volunteer to help figure that out Regards, Devananda 1: https://github.com/openstack/ironic/tree/master/ironic/nova/virt/ironic 2: https://review.openstack.org/95024 and https://review.openstack.org/95025 3: https://github.com/openstack/tempest/blob/master/tempest/scenario/test_baremetal_basic_ops.py 4: https://etherpad.openstack.org/p/juno-nova-deprecating-baremetal ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
