Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-22 Thread Sean Dague
On 07/15/2015 01:30 PM, Auld, Will wrote: Sean, OK, moving thread to openstack-dev. We'd like to help with this work if there is more to do. What are the next steps and what areas need help? Thanks, Will Thanks. Here is the current status: Grenade has been updated to have a

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-15 Thread Auld, Will
; Auld, Will; Troyer, Dean; mtrein...@kortar.org Subject: Re: [openstack-dev] [grenade] future direction on partial upgrade support Yes, they would. However, can we do this conversation on openstack-dev? It was started on an open list orginally, and that's where it should continue to evolve

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-06 Thread Armando M.
On 6 July 2015 at 13:13, Jeremy Stanley fu...@yuggoth.org wrote: On 2015-07-06 11:54:45 -0700 (-0700), Armando M. wrote: [...] For what I can tell, Joe was kind to set the infra to start gathering data on the reliability of the multi-node jobs, but they are clearly flaky [1], and

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-06 Thread Jeremy Stanley
On 2015-07-06 11:54:45 -0700 (-0700), Armando M. wrote: [...] For what I can tell, Joe was kind to set the infra to start gathering data on the reliability of the multi-node jobs, but they are clearly flaky [1], and currently broken. [...] Well, a check-.* pass rate of 25% is likely explained

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-06 Thread Armando M.
Hi, Not sure if we reached any conclusion with this thread, and I would like to resume it so that we don't derail the initial plan set forth by Russell and agreed during the Liberty summit, among other things. If I look at the thread I think this can be summarized as follow. Please correct me if

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-06 Thread Sean M. Collins
I'd also like to chime in - we've had some discussions on -infra today about the partial upgrade issue, and collected the following notes on an etherpad. https://etherpad.openstack.org/p/neutron-partial-upgrades One of the things identified, was the complexity of the DVR feature in Neutron, and

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-06 Thread Armando M.
Thanks Sean, comments inline. On 6 July 2015 at 16:58, Sean M. Collins s...@coreitpro.com wrote: I'd also like to chime in - we've had some discussions on -infra today about the partial upgrade issue, and collected the following notes on an etherpad.

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-07-06 Thread Anita Kuno
On 07/06/2015 09:02 PM, Armando M. wrote: Thanks Sean, comments inline. On 6 July 2015 at 16:58, Sean M. Collins s...@coreitpro.com wrote: I'd also like to chime in - we've had some discussions on -infra today about the partial upgrade issue, and collected the following notes on an

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-26 Thread Joe Gordon
On Wed, Jun 24, 2015 at 11:44 AM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jun 24, 2015 at 11:03 AM, Sean Dague s...@dague.net wrote: On 06/24/2015 01:41 PM, Russell Bryant wrote: On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-26 Thread Joe Gordon
No On Fri, Jun 26, 2015 at 10:15 AM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jun 24, 2015 at 11:44 AM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jun 24, 2015 at 11:03 AM, Sean Dague s...@dague.net wrote: On 06/24/2015 01:41 PM, Russell Bryant wrote: On 06/24/2015 01:31

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Joe Gordon
On Wed, Jun 24, 2015 at 10:45 AM, Sean Dague s...@dague.net wrote: On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Back when Nova first wanted to test partial upgrade, we did a bunch of

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Sean Dague
On 06/24/2015 01:41 PM, Russell Bryant wrote: On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Back when Nova first wanted to test partial upgrade, we did a bunch of slightly odd conditionals inside of

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Joe Gordon
On Wed, Jun 24, 2015 at 11:02 AM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jun 24, 2015 at 11:01 AM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jun 24, 2015 at 10:45 AM, Sean Dague s...@dague.net wrote: On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Joe Gordon
On Wed, Jun 24, 2015 at 11:01 AM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jun 24, 2015 at 10:45 AM, Sean Dague s...@dague.net wrote: On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Back

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Joe Gordon
On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague s...@dague.net wrote: Back when Nova first wanted to test partial upgrade, we did a bunch of slightly odd conditionals inside of grenade and devstack to make it so that if you were very careful, you could just not stop some of the old services on a

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Sean Dague
On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Back when Nova first wanted to test partial upgrade, we did a bunch of slightly odd conditionals inside of grenade and devstack to make it so

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Dan Smith
If multi-node isn't reliable more generally yet, do you think the simpler implementation of partial-upgrade testing could proceed? I've already done all of the patches to do it for Neutron. That way we could quickly get something in place to help block regressions and work on the

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Russell Bryant
On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Back when Nova first wanted to test partial upgrade, we did a bunch of slightly odd conditionals inside of grenade and devstack to make it so

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Russell Bryant
On 06/24/2015 01:49 PM, Dan Smith wrote: If multi-node isn't reliable more generally yet, do you think the simpler implementation of partial-upgrade testing could proceed? I've already done all of the patches to do it for Neutron. That way we could quickly get something in place to help

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-24 Thread Joe Gordon
On Wed, Jun 24, 2015 at 11:03 AM, Sean Dague s...@dague.net wrote: On 06/24/2015 01:41 PM, Russell Bryant wrote: On 06/24/2015 01:31 PM, Joe Gordon wrote: On Tue, Jun 16, 2015 at 9:58 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Back when Nova first wanted to test

Re: [openstack-dev] [grenade] future direction on partial upgrade support

2015-06-16 Thread Jeremy Stanley
On 2015-06-16 12:58:18 -0400 (-0400), Sean Dague wrote: [...] I think the only complexity here is the fact that grenade.sh implicitly drives stack.sh. Which means one of: 1) devstack-gate could build the worker first, then run grenade.sh 2) we make it so grenade.sh can execute in parts