Re: [openstack-dev] [magnum] versioned objects changes

[Mike Bayer]

On 8/26/15 4:47 AM, Grasza, Grzegorz wrote:


Hi,

I noticed that right now, when we make changes (adding/removing 
fields) in 
https://github.com/openstack/magnum/tree/master/magnum/objects , we 
don't change object versions.


The idea of objects is that each change in their fields should be 
versioned, documentation about the change should also be written in a 
comment inside the object and the obj_make_compatible method should be 
implemented or updated. See an example here:


https://github.com/openstack/nova/commit/ad6051bb5c2b62a0de6708cd2d7ac1e3cfd8f1d3#diff-7c6fefb09f0e1b446141d4c8f1ac5458L27

The question is, do you think magnum should support rolling upgrades 
from next release or maybe it's still too early?


If yes, I think core reviewers should start checking for these 
incompatible changes.


To clarify, rolling upgrades means support for running magnum services 
at different versions at the same time.


In Nova, there is an RPC call in the conductor to backport objects, 
which is called when older code gets an object it doesn’t understand. 
This patch does this in Magnum: https://review.openstack.org/#/c/184791/ .


I can report bugs and propose patches with version changes for this 
release, to get the effort started.


In Mitaka, when Grenade gets multi-node support, it can be used to add 
CI tests for rolling upgrades in Magnum.




from my POV I don't have strong feelings about using versioned objects 
early, however I do feel that the actual database migration model itself 
should *not* rely on expand/contract early in a project's lifecycle; 
expand/contract places severe limitations on the kinds of changes that 
can easily be made to a database schema and is better applied once the 
schema is mostly solidified in overall form, and the only changes that 
continue to be made are simple additions of new tables, columns and indexes.






/ Greg



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] versioned objects changes

[Egor Guz]

Adrian, agree with your points. But I think we should discuss it during the 
next team meeting and address/answer all concerns which team members may have. 
Grzegorz, can you join?

—
Egor

From: Adrian Otto mailto:adrian.o...@rackspace.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: Friday, August 28, 2015 at 18:51
To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [magnum] versioned objects changes

We are going to merge this work. I understand and respect Hongbin's position, 
but I respectfully disagree. When we are presented with ways to implement low 
overhead best practices like versioned objects, we will. It's not that hard to 
bump the version of an object when you change it. I like having systemic 
enforcement of that.

On the subject of review 211057, if you submit a review to remove comments, 
that is purely stylistic in nature, then you are inviting a discussion of style 
with our reviewers, and deserve to make that patch stylistically perfect.

If that patch had actual code in it tat made Magnum better, and several 
reviewers voted against the format of the comments, that would be stupid, and I 
would +2 it in spite of any -1 votes as long as it meets our rules for 
submission (like it must have a bug number).

Finally, meaningful -1 votes are valuable, and should not be viewed as a waste 
of effort. That's what we do as a team to help each other continually improve, 
and to make Magnum something we can all be proud of. With all that said, if you 
only have a stylistic comment, that should be a -0 vote with a comment, not a 
-1. If you are making stylistic and material comments together, that's fine, 
use a -1 vote.

Thanks,

Adrian

On Aug 28, 2015, at 5:21 PM, Davanum Srinivas 
mailto:dava...@gmail.com>> wrote:

Hongbin,

We are hearing the best advice available from the folks who started the 
library, evangelized it across nova, ironic, heat, neutron etc.

If we can spend so much time and energy (*FOUR* -1's on a review which just 
changes some commented lines - https://review.openstack.org/#/c/211057/) then 
we can and should clearly do better in things that really matter in the long 
run.

If we get into the rhythm of doing the right things and figuring out the steps 
needed right from the get go, it will pay off in the future.

My 2 cents.

Thanks,
Dims

PS: Note that i used "we" wearing my magnum core hat and not the o.vo/oslo core 
hat :)

On Fri, Aug 28, 2015 at 6:52 PM, Dan Smith 
mailto:d...@danplanet.com>> wrote:
> If you want my inexperienced opinion, a young project is the perfect
> time to start this.

^--- This ---^

> I understand that something like [2] will cause a test to fail when you
> make a major change to a versioned object. But you *want* that. It helps
> reviewers more easily catch contributors to say "You need to update the
> version, because the hash changed". The sooner you start using versioned
> objects in the way they are designed, the smaller the upfront cost, and
> it will also be a major savings later on if something like [1] pops up.

...and the way it will be the least overhead is if it's part of the
culture of contributors and reviewers. It's infinitely harder to take
the culture shift after everyone is used to not having to think about
upgrades, not to mention the technical recovery Ryan mentioned.

It's not my call for Magnum, but long-term thinking definitely pays off
in this particular area.

--Dan


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Davanum Srinivas :: https://twitter.com/dims
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] versioned objects changes

[Adrian Otto]

We are going to merge this work. I understand and respect Hongbin's position, 
but I respectfully disagree. When we are presented with ways to implement low 
overhead best practices like versioned objects, we will. It's not that hard to 
bump the version of an object when you change it. I like having systemic 
enforcement of that.

On the subject of review 211057, if you submit a review to remove comments, 
that is purely stylistic in nature, then you are inviting a discussion of style 
with our reviewers, and deserve to make that patch stylistically perfect.

If that patch had actual code in it tat made Magnum better, and several 
reviewers voted against the format of the comments, that would be stupid, and I 
would +2 it in spite of any -1 votes as long as it meets our rules for 
submission (like it must have a bug number).

Finally, meaningful -1 votes are valuable, and should not be viewed as a waste 
of effort. That's what we do as a team to help each other continually improve, 
and to make Magnum something we can all be proud of. With all that said, if you 
only have a stylistic comment, that should be a -0 vote with a comment, not a 
-1. If you are making stylistic and material comments together, that's fine, 
use a -1 vote.

Thanks,

Adrian

On Aug 28, 2015, at 5:21 PM, Davanum Srinivas 
mailto:dava...@gmail.com>> wrote:

Hongbin,

We are hearing the best advice available from the folks who started the 
library, evangelized it across nova, ironic, heat, neutron etc.

If we can spend so much time and energy (*FOUR* -1's on a review which just 
changes some commented lines - https://review.openstack.org/#/c/211057/) then 
we can and should clearly do better in things that really matter in the long 
run.

If we get into the rhythm of doing the right things and figuring out the steps 
needed right from the get go, it will pay off in the future.

My 2 cents.

Thanks,
Dims

PS: Note that i used "we" wearing my magnum core hat and not the o.vo/oslo core 
hat :)

On Fri, Aug 28, 2015 at 6:52 PM, Dan Smith 
mailto:d...@danplanet.com>> wrote:
> If you want my inexperienced opinion, a young project is the perfect
> time to start this.

^--- This ---^

> I understand that something like [2] will cause a test to fail when you
> make a major change to a versioned object. But you *want* that. It helps
> reviewers more easily catch contributors to say "You need to update the
> version, because the hash changed". The sooner you start using versioned
> objects in the way they are designed, the smaller the upfront cost, and
> it will also be a major savings later on if something like [1] pops up.

...and the way it will be the least overhead is if it's part of the
culture of contributors and reviewers. It's infinitely harder to take
the culture shift after everyone is used to not having to think about
upgrades, not to mention the technical recovery Ryan mentioned.

It's not my call for Magnum, but long-term thinking definitely pays off
in this particular area.

--Dan


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Davanum Srinivas :: https://twitter.com/dims
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] versioned objects changes

[Davanum Srinivas]

Hongbin,

We are hearing the best advice available from the folks who started the
library, evangelized it across nova, ironic, heat, neutron etc.

If we can spend so much time and energy (*FOUR* -1's on a review which just
changes some commented lines - https://review.openstack.org/#/c/211057/)
then we can and should clearly do better in things that really matter in
the long run.

If we get into the rhythm of doing the right things and figuring out the
steps needed right from the get go, it will pay off in the future.

My 2 cents.

Thanks,
Dims

PS: Note that i used "we" wearing my magnum core hat and not the o.vo/oslo
core hat :)

On Fri, Aug 28, 2015 at 6:52 PM, Dan Smith  wrote:

> > If you want my inexperienced opinion, a young project is the perfect
> > time to start this.
>
> ^--- This ---^
>
> > I understand that something like [2] will cause a test to fail when you
> > make a major change to a versioned object. But you *want* that. It helps
> > reviewers more easily catch contributors to say "You need to update the
> > version, because the hash changed". The sooner you start using versioned
> > objects in the way they are designed, the smaller the upfront cost, and
> > it will also be a major savings later on if something like [1] pops up.
>
> ...and the way it will be the least overhead is if it's part of the
> culture of contributors and reviewers. It's infinitely harder to take
> the culture shift after everyone is used to not having to think about
> upgrades, not to mention the technical recovery Ryan mentioned.
>
> It's not my call for Magnum, but long-term thinking definitely pays off
> in this particular area.
>
> --Dan
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Davanum Srinivas :: https://twitter.com/dims
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] versioned objects changes

[Dan Smith]

> If you want my inexperienced opinion, a young project is the perfect
> time to start this.

^--- This ---^

> I understand that something like [2] will cause a test to fail when you
> make a major change to a versioned object. But you *want* that. It helps
> reviewers more easily catch contributors to say "You need to update the
> version, because the hash changed". The sooner you start using versioned
> objects in the way they are designed, the smaller the upfront cost, and
> it will also be a major savings later on if something like [1] pops up.

...and the way it will be the least overhead is if it's part of the
culture of contributors and reviewers. It's infinitely harder to take
the culture shift after everyone is used to not having to think about
upgrades, not to mention the technical recovery Ryan mentioned.

It's not my call for Magnum, but long-term thinking definitely pays off
in this particular area.

--Dan


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] versioned objects changes

[Hongbin Lu]

Ryan,

Thanks for sharing your inputs. By looking through your response, I couldn't 
find the reasoning about why a young project is the perfect time to enforce a 
strict object version rule. I think a young project often starts with a static 
(or non-frequently changing) version until a point in time the project reaches 
a certain level of maturity. Isn't it? As a core reviewer of Magnum, I observe 
that the project is under fast development and objects are changing from time 
to time. It is very heavy to do all the work for strictly enforcing the version 
(bump version number, document the changed fields, re-generate the hashes, 
implement the compatibility check, etc.). Instead, I would prefer to let all 
objects stay in a beta version, until a time in future, the team decides to 
start bumping it.

Best regards,
Hongbin

From: Ryan Rossiter [mailto:rlros...@linux.vnet.ibm.com]
Sent: August-27-15 2:41 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [magnum] versioned objects changes

If you want my inexperienced opinion, a young project is the perfect time to 
start this. Nova has had a bunch of problems with versioned objects that don't 
get realized until the next release (because that's the point in time at which 
grenade (or worse, operators) catch this). At that point, you then need to hack 
things around and backport them in order to get them working in the old branch. 
[1] is an excellent example of Nova having to backport a fix to an object 
because we weren't using strict object testing.

I don't feel that this should be adding overhead to contributors and reviewers. 
With [2], this test absolutely helps both contributors and reviewers. Yes, it 
requires "fixing" things when a change happens to an object. Learning to do 
this "fix" to update object hashes is extremely easy to do and I hope my 
updated comment on there makes it even easier (also be aware I am new to 
OpenStack & Nova as of about 2 months ago, so this stuff was new to me too not 
very long ago).

I understand that something like [2] will cause a test to fail when you make a 
major change to a versioned object. But you *want* that. It helps reviewers 
more easily catch contributors to say "You need to update the version, because 
the hash changed". The sooner you start using versioned objects in the way they 
are designed, the smaller the upfront cost, and it will also be a major savings 
later on if something like [1] pops up.

[1]: https://bugs.launchpad.net/nova/+bug/1474074
[2]: https://review.openstack.org/#/c/217342/
On 8/27/2015 9:46 AM, Hongbin Lu wrote:
-1 from me.

IMHO, the rolling upgrade feature makes sense for a mature project (like Nova), 
but not for a young project like Magnum. It incurs overheads for contributors & 
reviewers to check the object compatibility in each patch. As you mentioned, 
the key benefit of this feature is supporting different version of magnum 
components running at the same time (i.e. running magnum-api 1.0 with 
magnum-conductor 1.1). I don't think supporting this advanced use case is a 
must at the current stage.

However, I don't mean to against merging patches of this feature. I just 
disagree to enforce the rule of object version change in the near future.

Best regards,
Hongbin

From: Grasza, Grzegorz [mailto:grzegorz.gra...@intel.com]
Sent: August-26-15 4:47 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [magnum] versioned objects changes

Hi,

I noticed that right now, when we make changes (adding/removing fields) in 
https://github.com/openstack/magnum/tree/master/magnum/objects , we don't 
change object versions.

The idea of objects is that each change in their fields should be versioned, 
documentation about the change should also be written in a comment inside the 
object and the obj_make_compatible method should be implemented or updated. See 
an example here:
https://github.com/openstack/nova/commit/ad6051bb5c2b62a0de6708cd2d7ac1e3cfd8f1d3#diff-7c6fefb09f0e1b446141d4c8f1ac5458L27

The question is, do you think magnum should support rolling upgrades from next 
release or maybe it's still too early?

If yes, I think core reviewers should start checking for these incompatible 
changes.

To clarify, rolling upgrades means support for running magnum services at 
different versions at the same time.
In Nova, there is an RPC call in the conductor to backport objects, which is 
called when older code gets an object it doesn't understand. This patch does 
this in Magnum: https://review.openstack.org/#/c/184791/ .

I can report bugs and propose patches with version changes for this release, to 
get the effort started.

In Mitaka, when Grenade gets multi-node support, it can be used to add CI tests 
for rolling upgrades in Magnum.


/ Greg





__

O

Re: [openstack-dev] [magnum] versioned objects changes

[Ryan Rossiter]

If you want my inexperienced opinion, a young project is the perfect 
time to start this. Nova has had a bunch of problems with versioned 
objects that don't get realized until the next release (because that's 
the point in time at which grenade (or worse, operators) catch this). At 
that point, you then need to hack things around and backport them in 
order to get them working in the old branch. [1] is an excellent example 
of Nova having to backport a fix to an object because we weren't using 
strict object testing.


I don't feel that this should be adding overhead to contributors and 
reviewers. With [2], this test absolutely helps both contributors and 
reviewers. Yes, it requires "fixing" things when a change happens to an 
object. Learning to do this "fix" to update object hashes is extremely 
easy to do and I hope my updated comment on there makes it even easier 
(also be aware I am new to OpenStack & Nova as of about 2 months ago, so 
this stuff was new to me too not very long ago).


I understand that something like [2] will cause a test to fail when you 
make a major change to a versioned object. But you *want* that. It helps 
reviewers more easily catch contributors to say "You need to update the 
version, because the hash changed". The sooner you start using versioned 
objects in the way they are designed, the smaller the upfront cost, and 
it will also be a major savings later on if something like [1] pops up.


[1]: https://bugs.launchpad.net/nova/+bug/1474074
[2]: https://review.openstack.org/#/c/217342/

On 8/27/2015 9:46 AM, Hongbin Lu wrote:


-1 from me.

IMHO, the rolling upgrade feature makes sense for a mature project 
(like Nova), but not for a young project like Magnum. It incurs 
overheads for contributors & reviewers to check the object 
compatibility in each patch. As you mentioned, the key benefit of this 
feature is supporting different version of magnum components running 
at the same time (i.e. running magnum-api 1.0 with magnum-conductor 
1.1). I don’t think supporting this advanced use case is a must at the 
current stage.


However, I don’t mean to against merging patches of this feature. I 
just disagree to enforce the rule of object version change in the near 
future.


Best regards,

Hongbin

*From:*Grasza, Grzegorz [mailto:grzegorz.gra...@intel.com]
*Sent:* August-26-15 4:47 AM
*To:* OpenStack Development Mailing List (not for usage questions)
*Subject:* [openstack-dev] [magnum] versioned objects changes

Hi,

I noticed that right now, when we make changes (adding/removing 
fields) in 
https://github.com/openstack/magnum/tree/master/magnum/objects , we 
don't change object versions.


The idea of objects is that each change in their fields should be 
versioned, documentation about the change should also be written in a 
comment inside the object and the obj_make_compatible method should be 
implemented or updated. See an example here:


https://github.com/openstack/nova/commit/ad6051bb5c2b62a0de6708cd2d7ac1e3cfd8f1d3#diff-7c6fefb09f0e1b446141d4c8f1ac5458L27

The question is, do you think magnum should support rolling upgrades 
from next release or maybe it's still too early?


If yes, I think core reviewers should start checking for these 
incompatible changes.


To clarify, rolling upgrades means support for running magnum services 
at different versions at the same time.


In Nova, there is an RPC call in the conductor to backport objects, 
which is called when older code gets an object it doesn’t understand. 
This patch does this in Magnum: https://review.openstack.org/#/c/184791/ .


I can report bugs and propose patches with version changes for this 
release, to get the effort started.


In Mitaka, when Grenade gets multi-node support, it can be used to add 
CI tests for rolling upgrades in Magnum.


/ Greg



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Thanks,

Ryan Rossiter

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] versioned objects changes

[Hongbin Lu]

-1 from me.

IMHO, the rolling upgrade feature makes sense for a mature project (like Nova), 
but not for a young project like Magnum. It incurs overheads for contributors & 
reviewers to check the object compatibility in each patch. As you mentioned, 
the key benefit of this feature is supporting different version of magnum 
components running at the same time (i.e. running magnum-api 1.0 with 
magnum-conductor 1.1). I don't think supporting this advanced use case is a 
must at the current stage.

However, I don't mean to against merging patches of this feature. I just 
disagree to enforce the rule of object version change in the near future.

Best regards,
Hongbin

From: Grasza, Grzegorz [mailto:grzegorz.gra...@intel.com]
Sent: August-26-15 4:47 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [magnum] versioned objects changes

Hi,

I noticed that right now, when we make changes (adding/removing fields) in 
https://github.com/openstack/magnum/tree/master/magnum/objects , we don't 
change object versions.

The idea of objects is that each change in their fields should be versioned, 
documentation about the change should also be written in a comment inside the 
object and the obj_make_compatible method should be implemented or updated. See 
an example here:
https://github.com/openstack/nova/commit/ad6051bb5c2b62a0de6708cd2d7ac1e3cfd8f1d3#diff-7c6fefb09f0e1b446141d4c8f1ac5458L27

The question is, do you think magnum should support rolling upgrades from next 
release or maybe it's still too early?

If yes, I think core reviewers should start checking for these incompatible 
changes.

To clarify, rolling upgrades means support for running magnum services at 
different versions at the same time.
In Nova, there is an RPC call in the conductor to backport objects, which is 
called when older code gets an object it doesn't understand. This patch does 
this in Magnum: https://review.openstack.org/#/c/184791/ .

I can report bugs and propose patches with version changes for this release, to 
get the effort started.

In Mitaka, when Grenade gets multi-node support, it can be used to add CI tests 
for rolling upgrades in Magnum.


/ Greg

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] versioned objects changes

[Ryan Rossiter]

I've been working with Nova's versioned objects lately to help catch 
people when object changes are made. There is a lot of object-related 
tests in Nova for this, and a major one I can see helping this situation 
is this test [1]. Looking through the different versioned objects within 
Magnum, I don't see any objects that hold subobjects, so tests like [2] 
are not really necessary yet.


I have uploaded a review for bringing [1] from Nova into Magnum [3]. I 
think this will be a major step in the right direction towards keeping 
track of object changes that will help with rolling upgrades.


[1]: 
https://github.com/openstack/nova/blob/master/nova/tests/unit/objects/test_objects.py#L1262-L1286
[2]: 
https://github.com/openstack/nova/blob/master/nova/tests/unit/objects/test_objects.py#L1314

[3]: https://review.openstack.org/#/c/217342/

On 8/26/2015 3:47 AM, Grasza, Grzegorz wrote:


Hi,

I noticed that right now, when we make changes (adding/removing 
fields) in 
https://github.com/openstack/magnum/tree/master/magnum/objects , we 
don't change object versions.


The idea of objects is that each change in their fields should be 
versioned, documentation about the change should also be written in a 
comment inside the object and the obj_make_compatible method should be 
implemented or updated. See an example here:


https://github.com/openstack/nova/commit/ad6051bb5c2b62a0de6708cd2d7ac1e3cfd8f1d3#diff-7c6fefb09f0e1b446141d4c8f1ac5458L27

The question is, do you think magnum should support rolling upgrades 
from next release or maybe it's still too early?


If yes, I think core reviewers should start checking for these 
incompatible changes.


To clarify, rolling upgrades means support for running magnum services 
at different versions at the same time.


In Nova, there is an RPC call in the conductor to backport objects, 
which is called when older code gets an object it doesn’t understand. 
This patch does this in Magnum: https://review.openstack.org/#/c/184791/ .


I can report bugs and propose patches with version changes for this 
release, to get the effort started.


In Mitaka, when Grenade gets multi-node support, it can be used to add 
CI tests for rolling upgrades in Magnum.


/ Greg



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Thanks,

Ryan Rossiter

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev