Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
On Nov 24, 2014, at 4:06 PM, Jay Pipes jaypi...@gmail.com wrote: On 11/24/2014 03:11 PM, Joshua Harlow wrote: Dan Smith wrote: 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. After some discussion with some of the interested parties, we're planning to add a third .z element to the version numbers and use that to handle backports in the same way that we do for RPC: https://review.openstack.org/#/c/134623/ Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? I'm not planning to look into this, especially since we discussed it a couple years ago when deciding to do what we're currently doing. If someone else does, creates a thing that is demonstrably more useful than what we have, and provides a migration plan, then cool. Otherwise, I'm not really planning to stop what I'm doing at the moment. - Are there any other concrete things we can do to get this usable by other projects in a timely manner? To be honest, since the summit, I've not done anything with the current oslo spec, given the potential for doing something different that was raised. I know that cinder folks (at least) are planning to start copying code into their tree to get moving. I think we need a decision to either (a) dump what we've got into the proposed library (or incubator) and plan to move forward incrementally or (b) each continue doing our own thing(s) in our own trees while we wait for someone to create something based on GPB that does what we want. I'd prefer (a); although I hope there is a owner/lead for this library (dan?) and it's not just dumped on the oslo folks as that won't work out so well I think. It'd be nice if said owner could also look into (b) but that's at there own (or other library supporter) time I suppose (I personally think (b) would probably allow for a larger community of folks to get involved in this library, would potentially reduce the amount of custom/overlapping code and other similar benefits...). I gave some comments at the very end of the summit session on this, and I want to be clear about something. I definitely like GPB, and there's definite overlap with some things that GPB does and things that nova.objects does. That said, I don't think it's wise to make oslo-versionedobjects be a totally new thing. I think we should use nova.objects as the base of a new oslo-versionedobjects library, and we should evolve oslo-versionedobjects slowly over time, eventually allowing for nova, ironic, and whomever else is currently using nova/objects, to align with an Oslo library vision for this. So, in short, I also think a) is the appropriate path to take. +1 When Dan and I talked about this, I said I would take care of exporting the nova objects git history into a new repository. We’ve had some other things blocking work in Oslo that I needed to handle, but I expect to be able to pick up this work soon. If someone else wants to jump in, I’ll be happy to give a brain dump of what I planned to do for the export, since the existing Oslo tool that we use on the incubator isn’t quite right for the job. Doug Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
On 11/13/2014 01:12 AM, Robert Collins wrote: On 11 November 2014 13:30, Angus Salkeld asalk...@mirantis.com wrote: Hi all I just wanted to make sure we are all under the same understanding of the outcomes and what the next steps for the versioned objects session are. 1. There is a lot of interest in other projects using oslo versioned objects and it is worth progressing with this (https://review.openstack.org/#/c/127532). 2. jpipes and jharlow suggested experimenting/investigating google protocol buffers (https://developers.google.com/protocol-buffers/) instead of the custom serialization and version code. This *could* be an implementation detail, but also could make the adoption by nova more complicated (as it has a different mechanism in place). 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? - Are there any other concrete things we can do to get this usable by other projects in a timely manner? +1 on protocol buffers, but perhaps http://kentonv.github.io/capnproto/ could be considered: its protocol buffers v2 basically - from one of the originators of protocol buffers. It has Python support available too, just like protocol buffers. Very nice indeed. Been reading through the Cap'n'proto documentation and it looks like a great improvement over GPB. Definitely something to look into. I sent an email privately to Angus and Dan this morning saying that I personally don't have the bandwidth to do a PoC that would use GPB as implementation of the serialization, schema representation, and versioning engine. I support the idea of using GPB, but I also recognize it's a large amount of work. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. After some discussion with some of the interested parties, we're planning to add a third .z element to the version numbers and use that to handle backports in the same way that we do for RPC: https://review.openstack.org/#/c/134623/ Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? I'm not planning to look into this, especially since we discussed it a couple years ago when deciding to do what we're currently doing. If someone else does, creates a thing that is demonstrably more useful than what we have, and provides a migration plan, then cool. Otherwise, I'm not really planning to stop what I'm doing at the moment. - Are there any other concrete things we can do to get this usable by other projects in a timely manner? To be honest, since the summit, I've not done anything with the current oslo spec, given the potential for doing something different that was raised. I know that cinder folks (at least) are planning to start copying code into their tree to get moving. I think we need a decision to either (a) dump what we've got into the proposed library (or incubator) and plan to move forward incrementally or (b) each continue doing our own thing(s) in our own trees while we wait for someone to create something based on GPB that does what we want. --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
Dan Smith wrote: 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. After some discussion with some of the interested parties, we're planning to add a third .z element to the version numbers and use that to handle backports in the same way that we do for RPC: https://review.openstack.org/#/c/134623/ Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? I'm not planning to look into this, especially since we discussed it a couple years ago when deciding to do what we're currently doing. If someone else does, creates a thing that is demonstrably more useful than what we have, and provides a migration plan, then cool. Otherwise, I'm not really planning to stop what I'm doing at the moment. - Are there any other concrete things we can do to get this usable by other projects in a timely manner? To be honest, since the summit, I've not done anything with the current oslo spec, given the potential for doing something different that was raised. I know that cinder folks (at least) are planning to start copying code into their tree to get moving. I think we need a decision to either (a) dump what we've got into the proposed library (or incubator) and plan to move forward incrementally or (b) each continue doing our own thing(s) in our own trees while we wait for someone to create something based on GPB that does what we want. I'd prefer (a); although I hope there is a owner/lead for this library (dan?) and it's not just dumped on the oslo folks as that won't work out so well I think. It'd be nice if said owner could also look into (b) but that's at there own (or other library supporter) time I suppose (I personally think (b) would probably allow for a larger community of folks to get involved in this library, would potentially reduce the amount of custom/overlapping code and other similar benefits...). --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
On 11/24/2014 03:11 PM, Joshua Harlow wrote: Dan Smith wrote: 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. After some discussion with some of the interested parties, we're planning to add a third .z element to the version numbers and use that to handle backports in the same way that we do for RPC: https://review.openstack.org/#/c/134623/ Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? I'm not planning to look into this, especially since we discussed it a couple years ago when deciding to do what we're currently doing. If someone else does, creates a thing that is demonstrably more useful than what we have, and provides a migration plan, then cool. Otherwise, I'm not really planning to stop what I'm doing at the moment. - Are there any other concrete things we can do to get this usable by other projects in a timely manner? To be honest, since the summit, I've not done anything with the current oslo spec, given the potential for doing something different that was raised. I know that cinder folks (at least) are planning to start copying code into their tree to get moving. I think we need a decision to either (a) dump what we've got into the proposed library (or incubator) and plan to move forward incrementally or (b) each continue doing our own thing(s) in our own trees while we wait for someone to create something based on GPB that does what we want. I'd prefer (a); although I hope there is a owner/lead for this library (dan?) and it's not just dumped on the oslo folks as that won't work out so well I think. It'd be nice if said owner could also look into (b) but that's at there own (or other library supporter) time I suppose (I personally think (b) would probably allow for a larger community of folks to get involved in this library, would potentially reduce the amount of custom/overlapping code and other similar benefits...). I gave some comments at the very end of the summit session on this, and I want to be clear about something. I definitely like GPB, and there's definite overlap with some things that GPB does and things that nova.objects does. That said, I don't think it's wise to make oslo-versionedobjects be a totally new thing. I think we should use nova.objects as the base of a new oslo-versionedobjects library, and we should evolve oslo-versionedobjects slowly over time, eventually allowing for nova, ironic, and whomever else is currently using nova/objects, to align with an Oslo library vision for this. So, in short, I also think a) is the appropriate path to take. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
On Nov 24, 2014, at 13:06, Jay Pipes jaypi...@gmail.com wrote: That said, I don't think it's wise to make oslo-versionedobjects be a totally new thing. I think we should use nova.objects as the base of a new oslo-versionedobjects library, and we should evolve oslo-versionedobjects slowly over time, eventually allowing for nova, ironic, and whomever else is currently using nova/objects, to align with an Oslo library vision for this. So, in short, I also think a) is the appropriate path to take. +1 I'd like to see oslo-versionedobjects start out with nova.objects as the implementation, with the ability to add support for protobuf later. melanie (melwitt) signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
On Tue, Nov 25, 2014 at 7:06 AM, Jay Pipes jaypi...@gmail.com wrote: On 11/24/2014 03:11 PM, Joshua Harlow wrote: Dan Smith wrote: 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. After some discussion with some of the interested parties, we're planning to add a third .z element to the version numbers and use that to handle backports in the same way that we do for RPC: https://review.openstack.org/#/c/134623/ Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? I'm not planning to look into this, especially since we discussed it a couple years ago when deciding to do what we're currently doing. If someone else does, creates a thing that is demonstrably more useful than what we have, and provides a migration plan, then cool. Otherwise, I'm not really planning to stop what I'm doing at the moment. - Are there any other concrete things we can do to get this usable by other projects in a timely manner? To be honest, since the summit, I've not done anything with the current oslo spec, given the potential for doing something different that was raised. I know that cinder folks (at least) are planning to start copying code into their tree to get moving. I think we need a decision to either (a) dump what we've got into the proposed library (or incubator) and plan to move forward incrementally or (b) each continue doing our own thing(s) in our own trees while we wait for someone to create something based on GPB that does what we want. I'd prefer (a); although I hope there is a owner/lead for this library (dan?) and it's not just dumped on the oslo folks as that won't work out so well I think. It'd be nice if said owner could also look into (b) but that's at there own (or other library supporter) time I suppose (I personally think (b) would probably allow for a larger community of folks to get involved in this library, would potentially reduce the amount of custom/overlapping code and other similar benefits...). I gave some comments at the very end of the summit session on this, and I want to be clear about something. I definitely like GPB, and there's definite overlap with some things that GPB does and things that nova.objects does. That said, I don't think it's wise to make oslo-versionedobjects be a totally new thing. I think we should use nova.objects as the base of a new oslo-versionedobjects library, and we should evolve oslo-versionedobjects slowly over time, eventually allowing for nova, ironic, and whomever else is currently using nova/objects, to align with an Oslo library vision for this. So, in short, I also think a) is the appropriate path to take. Yeah, my concern with (b) is the time it will take for other projects to get to use it, esp. since no one is jumping to take the work on. -Angus Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
On 11 November 2014 13:30, Angus Salkeld asalk...@mirantis.com wrote: Hi all I just wanted to make sure we are all under the same understanding of the outcomes and what the next steps for the versioned objects session are. 1. There is a lot of interest in other projects using oslo versioned objects and it is worth progressing with this (https://review.openstack.org/#/c/127532). 2. jpipes and jharlow suggested experimenting/investigating google protocol buffers (https://developers.google.com/protocol-buffers/) instead of the custom serialization and version code. This *could* be an implementation detail, but also could make the adoption by nova more complicated (as it has a different mechanism in place). 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? - Are there any other concrete things we can do to get this usable by other projects in a timely manner? +1 on protocol buffers, but perhaps http://kentonv.github.io/capnproto/ could be considered: its protocol buffers v2 basically - from one of the originators of protocol buffers. It has Python support available too, just like protocol buffers. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
Neat, Didn't know that existed. http://kentonv.github.io/capnproto/language.html does look pretty nice... -Josh From: Robert Collins robe...@robertcollins.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, November 12, 2014 10:12 PM Subject: Re: [openstack-dev] [all] Versioned objects cross project sessions next steps On 11 November 2014 13:30, Angus Salkeld asalk...@mirantis.com wrote: Hi all I just wanted to make sure we are all under the same understanding of the outcomes and what the next steps for the versioned objects session are. 1. There is a lot of interest in other projects using oslo versioned objects and it is worth progressing with this (https://review.openstack.org/#/c/127532). 2. jpipes and jharlow suggested experimenting/investigating google protocol buffers (https://developers.google.com/protocol-buffers/) instead of the custom serialization and version code. This *could* be an implementation detail, but also could make the adoption by nova more complicated (as it has a different mechanism in place). 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? - Are there any other concrete things we can do to get this usable by other projects in a timely manner? +1 on protocol buffers, but perhaps http://kentonv.github.io/capnproto/ could be considered: its protocol buffers v2 basically - from one of the originators of protocol buffers. It has Python support available too, just like protocol buffers. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Versioned objects cross project sessions next steps
On Nov 10, 2014, at 4:30 PM, Angus Salkeld asalk...@mirantis.com wrote: Hi all I just wanted to make sure we are all under the same understanding of the outcomes and what the next steps for the versioned objects session are. 1. There is a lot of interest in other projects using oslo versioned objects and it is worth progressing with this (https://review.openstack.org/#/c/127532 https://review.openstack.org/#/c/127532). 2. jpipes and jharlow suggested experimenting/investigating google protocol buffers (https://developers.google.com/protocol-buffers/) https://developers.google.com/protocol-buffers/%29 instead of the custom serialization and version code. This *could* be an implementation detail, but also could make the adoption by nova more complicated (as it has a different mechanism in place). 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? - Are there any other concrete things we can do to get this usable by other projects in a timely manner? Regards Angus I am personally very interested in seeing what the spec for adopting protobuf ends up looking like in comparison to the current Nova implementation. The main reason is that there is already a fairly well driven mechanism in place to utilize the objects across different languages. This would allow for better consumption of any/all notifications from outside of the OpenStack ecosystem as well as cases (such as tokens) that might legitimately need to evolve the format in the longer-term without breaking *everything*. I also think that it would largely be possible to provide a way to wrap the current Nova object model into protobuf as a conversion/transition plan. This doesn’t discount that protobuf may in-fact not be a good fit for our use cases and the Nova model does have some reasonable drive-time on it. It very well might be a good solution for all the projects. Regarding stable patches, largely if the object is versioned differently it would be more difficult to back port a change. I, however, am going to hazard a guess that most of the issues are not directly revolving around the serialization/deserialization code (nor the data directly driven by it) meaning that the incremental difference in back port headaches would be relatively minor. In short, I think breaking the hard-tie between the underlying storage model and the in-memory/serialization mechanism would be good for all of the projects. Jay/Dan/Angus, please let me know if I can be any help with exploring protobuf. Cheers, Morgan___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] Versioned objects cross project sessions next steps
Hi all I just wanted to make sure we are all under the same understanding of the outcomes and what the next steps for the versioned objects session are. 1. There is a lot of interest in other projects using oslo versioned objects and it is worth progressing with this ( https://review.openstack.org/#/c/127532). 2. jpipes and jharlow suggested experimenting/investigating google protocol buffers (https://developers.google.com/protocol-buffers/) instead of the custom serialization and version code. This *could* be an implementation detail, but also could make the adoption by nova more complicated (as it has a different mechanism in place). 3. vish brought up one draw back of versioned objects: the difficulty in cherry picking commits for stable branches - Is this a show stopper?. Next steps: - Jay suggested making a second spec that would lay out what it would look like if we used google protocol buffers. - Dan: do you need some help in making this happen, do we need some volunteers? - Are there any other concrete things we can do to get this usable by other projects in a timely manner? Regards Angus ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
