Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Lets use https://etherpad.openstack.org/p/Icehouse-nova-oslo-sync to keep track of things. On Wed, Feb 26, 2014 at 5:10 PM, Joe Gordon joe.gord...@gmail.com wrote: GCB, Ben, Thanks for volunteering to help. GCB, reminded me that we should be doing this for python-novaclient in addition to nova itself. So that being said, as I see it here are the steps moving forward: Note: as previously mentionedin this thread there already is a team working on syncing oslo.db so we can ignore that for now (and once its ready they will propose patches, so we just have to do reviews). 1) Review all outstanding nova/python-novaclient sync patches. https://review.openstack.org/#/c/72596/ https://review.openstack.org/#/c/74560/ https://review.openstack.org/#/c/75644/ 2) Using update.sh sync all low hanging fruit in both repos all at once. Low hanging fruit is anything that doesn't require a change outside of */openstack/common. As usual when doing these syncs we should list all patches being synced across, as well as document which modules we aren't syncing accross ./update.sh --base novaclient --config-file ../python-novaclient/openstack-common.conf --dest-dir ../python-novaclient/ https://review.openstack.org/#/c/76642/ ./update.sh --base nova --config-file ../nova/openstack-common.conf --dest-dir ../nova/ 3) At this point we should have a list of modules that are non-trivial to sync, now we can triage them and decide if they are oslo bugs or if nova/python-novaclient code needs updating. So for now we need reviews on the patches listed in 1, and someone to work on the low hanging fruit sync for nova. Followed by triaging of the non-low hanging fruit. Once we have the low hanging fruit out of the way lets sync up about how to handle the rest. best, Joe On Fri, Feb 21, 2014 at 6:26 PM, ChangBo Guo glongw...@gmail.com wrote: 2014-02-22 5:09 GMT+08:00 Ben Nemec openst...@nemebean.com: /me finally catches up on -dev list traffic... On 2014-02-19 20:27, Doug Hellmann wrote: On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. Thank you for volunteering to spear-head this, Joe. +1 To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? I will help, especially with reviews and tracking. I'm happy to help as well. I always try to help with oslo syncs any time I'm made aware of problems anyway. What is our first step here? Get the low-hanging fruit syncs proposed all at once? Do them individually (taking into consideration the module deps, of course)? If we're going to try to get this done for Icehouse then we probably need to start ASAP. -Ben I also would like to be volunteer of the new team :) -gcb -- ChangBo Guo(gcb) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
GCB, Ben, Thanks for volunteering to help. GCB, reminded me that we should be doing this for python-novaclient in addition to nova itself. So that being said, as I see it here are the steps moving forward: Note: as previously mentionedin this thread there already is a team working on syncing oslo.db so we can ignore that for now (and once its ready they will propose patches, so we just have to do reviews). 1) Review all outstanding nova/python-novaclient sync patches. https://review.openstack.org/#/c/72596/ https://review.openstack.org/#/c/74560/ https://review.openstack.org/#/c/75644/ 2) Using update.sh sync all low hanging fruit in both repos all at once. Low hanging fruit is anything that doesn't require a change outside of */openstack/common. As usual when doing these syncs we should list all patches being synced across, as well as document which modules we aren't syncing accross ./update.sh --base novaclient --config-file ../python-novaclient/openstack-common.conf --dest-dir ../python-novaclient/ https://review.openstack.org/#/c/76642/ ./update.sh --base nova --config-file ../nova/openstack-common.conf --dest-dir ../nova/ 3) At this point we should have a list of modules that are non-trivial to sync, now we can triage them and decide if they are oslo bugs or if nova/python-novaclient code needs updating. So for now we need reviews on the patches listed in 1, and someone to work on the low hanging fruit sync for nova. Followed by triaging of the non-low hanging fruit. Once we have the low hanging fruit out of the way lets sync up about how to handle the rest. best, Joe On Fri, Feb 21, 2014 at 6:26 PM, ChangBo Guo glongw...@gmail.com wrote: 2014-02-22 5:09 GMT+08:00 Ben Nemec openst...@nemebean.com: /me finally catches up on -dev list traffic... On 2014-02-19 20:27, Doug Hellmann wrote: On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. Thank you for volunteering to spear-head this, Joe. +1 To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? I will help, especially with reviews and tracking. I'm happy to help as well. I always try to help with oslo syncs any time I'm made aware of problems anyway. What is our first step here? Get the low-hanging fruit syncs proposed all at once? Do them individually (taking into consideration the module deps, of course)? If we're going to try to get this done for Icehouse then we probably need to start ASAP. -Ben I also would like to be volunteer of the new team :) -gcb -- ChangBo Guo(gcb) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
/me finally catches up on -dev list traffic... On 2014-02-19 20:27, Doug Hellmann wrote: On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. Thank you for volunteering to spear-head this, Joe. +1 To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? I will help, especially with reviews and tracking. I'm happy to help as well. I always try to help with oslo syncs any time I'm made aware of problems anyway. What is our first step here? Get the low-hanging fruit syncs proposed all at once? Do them individually (taking into consideration the module deps, of course)? If we're going to try to get this done for Icehouse then we probably need to start ASAP. -Ben We are going to want someone from the team working on the db modules to participate as well, since we know that's one area where the API has diverged some (although we did take backwards compatibility into account). Victor, can you help find us a volunteer? Doug best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev [1] ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev [1] Links: -- [1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
2014-02-22 5:09 GMT+08:00 Ben Nemec openst...@nemebean.com: /me finally catches up on -dev list traffic... On 2014-02-19 20:27, Doug Hellmann wrote: On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. Thank you for volunteering to spear-head this, Joe. +1 To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? I will help, especially with reviews and tracking. I'm happy to help as well. I always try to help with oslo syncs any time I'm made aware of problems anyway. What is our first step here? Get the low-hanging fruit syncs proposed all at once? Do them individually (taking into consideration the module deps, of course)? If we're going to try to get this done for Icehouse then we probably need to start ASAP. -Ben I also would like to be volunteer of the new team :) -gcb -- ChangBo Guo(gcb) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Hi all, I'm ready to help with syncing of DB code. But we'll need reviewers attention in both oslo-incubator in nova :) Thanks, Roman On Thu, Feb 20, 2014 at 5:37 AM, Lance D Bragstad ldbra...@us.ibm.com wrote: Shed a little bit of light on Matt's comment about Keystone removing oslo-incubator code and the issues we hit. Comments below. Best Regards, Lance Bragstad ldbra...@us.ibm.com Doug Hellmann doug.hellm...@dreamhost.com wrote on 02/19/2014 09:00:29 PM: From: Doug Hellmann doug.hellm...@dreamhost.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 02/19/2014 09:12 PM Subject: Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon joe.gord...@gmail.com wrote: As a side to this, as an exercise I tried a oslo sync in cinder to see what kind of issues would arise and here are my findings so far: https://review.openstack.org/#/c/74786/ On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: On 2/19/2014 7:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Well I'll get the ball rolling... In the past when this has come up there is always a debate over should be just sync to sync because we should always be up to date, or is that dangerous and we should only sync when there is a need (which is what the review guidelines say now [1]). There are pros and cons: pros: - we get bug fixes that we didn't know existed - it should be less painful to sync if we do it more often cons: - it's more review overhead and some crazy guy thinks we need a special team dedicated to reviewing those changes :) - there are some changes in o-i that would break nova; I'm specifically thinking of the oslo RequestContext which has domain support now (or some other keystone thingy) and nova has it's own RequestContext - so if we did sync that from o-i it would change nova's logging context and break on us since we didn't use oslo context. For that last con, I'd argue that we should move to the oslo RequestContext, I'm not sure why we aren't. Would that module then not fall under low-hanging-fruit? I am classifying low hanging fruit as anything that doesn't require any nova changes to work. +1 I think the DB API modules have been a concern for auto-syncing before too but I can't remember why now...something about possibly changing the behavior of how the nova migrations would work? But if they are already using the common code, I don't see the issue. AFAIK there is already a team working on db api syncing, so I was thinking of let them deal with it. +1 Doug This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? I think the keystone guys are running into a similar issue where they want to remove a bunch of now-dead messaging code from keystone but can't because there are still some things in oslo-incubator using oslo.messaging code, or something weird like that. So maybe those
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Hello All I and Roman Podoliaka are familiar with the changes made to common db code, so we are ready to help with syncing it to OS projects. But we want to ask you for more activity in reviewing of these patches. Thanks, Victor On Thu, Feb 20, 2014 at 4:27 AM, Doug Hellmann doug.hellm...@dreamhost.comwrote: On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. Thank you for volunteering to spear-head this, Joe. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? I will help, especially with reviews and tracking. We are going to want someone from the team working on the db modules to participate as well, since we know that's one area where the API has diverged some (although we did take backwards compatibility into account). Victor, can you help find us a volunteer? Doug best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Matt Riedemann wrote: This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? FWIW the code was recently removed from the oslo-incubator, once Neutron (the last of the rootwrap-consuming projects) got migrated to using oslo.rootwrap. [2] https://review.openstack.org/#/c/73340/ This one syncs changes from https://review.openstack.org/#/c/63094 63094 should never have been approved, since rootwrap in oslo-incubator was frozen (graduating). Now the changes are lost, since they were never proposed to oslo.rootwrap, and the code in the incubator was cleaned up. I'll comment on the 73340 review to try to solve this mess. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On 2/19/2014 7:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Well I'll get the ball rolling... In the past when this has come up there is always a debate over should be just sync to sync because we should always be up to date, or is that dangerous and we should only sync when there is a need (which is what the review guidelines say now [1]). There are pros and cons: pros: - we get bug fixes that we didn't know existed - it should be less painful to sync if we do it more often cons: - it's more review overhead and some crazy guy thinks we need a special team dedicated to reviewing those changes :) - there are some changes in o-i that would break nova; I'm specifically thinking of the oslo RequestContext which has domain support now (or some other keystone thingy) and nova has it's own RequestContext - so if we did sync that from o-i it would change nova's logging context and break on us since we didn't use oslo context. For that last con, I'd argue that we should move to the oslo RequestContext, I'm not sure why we aren't. Would that module then not fall under low-hanging-fruit? I think the DB API modules have been a concern for auto-syncing before too but I can't remember why now...something about possibly changing the behavior of how the nova migrations would work? But if they are already using the common code, I don't see the issue. This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? I think the keystone guys are running into a similar issue where they want to remove a bunch of now-dead messaging code from keystone but can't because there are still some things in oslo-incubator using oslo.messaging code, or something weird like that. So maybe those modules are considered out of scope for this effort until the o-r/o-m code is completely out of o-i? Finally, just like we'd like to have cores for each virt driver in nova and the neutron API in nova, I think this kind of thing, at least initially, would benefit from having some oslo cores involved in a team that are also familiar to a degree with nova, e.g. bnemec or dims. [1] https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist [2] https://review.openstack.org/#/c/73340/ -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. Thank you for volunteering to spear-head this, Joe. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? I will help, especially with reviews and tracking. We are going to want someone from the team working on the db modules to participate as well, since we know that's one area where the API has diverged some (although we did take backwards compatibility into account). Victor, can you help find us a volunteer? Doug best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On Wed, Feb 19, 2014 at 9:20 PM, Matt Riedemann mrie...@linux.vnet.ibm.comwrote: On 2/19/2014 7:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Well I'll get the ball rolling... In the past when this has come up there is always a debate over should be just sync to sync because we should always be up to date, or is that dangerous and we should only sync when there is a need (which is what the review guidelines say now [1]). There are pros and cons: pros: - we get bug fixes that we didn't know existed - it should be less painful to sync if we do it more often cons: - it's more review overhead and some crazy guy thinks we need a special team dedicated to reviewing those changes :) - there are some changes in o-i that would break nova; I'm specifically thinking of the oslo RequestContext which has domain support now (or some other keystone thingy) and nova has it's own RequestContext - so if we did sync that from o-i it would change nova's logging context and break on us since we didn't use oslo context. Another con is that if we do find a critical bug in an incubator module, and a project that uses that module is far out of date, applying the fix may be more difficult. (This is also another motivation for moving code out of the incubator entirely, but as Joe pointed out earlier today, that's not really a short-term solution.) For that last con, I'd argue that we should move to the oslo RequestContext, I'm not sure why we aren't. Would that module then not fall under low-hanging-fruit? I think the DB API modules have been a concern for auto-syncing before too but I can't remember why now...something about possibly changing the behavior of how the nova migrations would work? But if they are already using the common code, I don't see the issue. There has been some recent work on the db code to make it more suitable for use in some of the other projects that don't have a single global session pool. There's a compatibility shim, which should make the update painless, but it's not just a simple file copy. This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? After a module graduates to a library, we treat the incubator copy as the stable branch until all of the integrated projects that consume the module have migrated to the new library. That way if bugs are found, the fixes can be applied to a project without having to also migrate to the library. So, the best action is to port to the library. As a fall back, at least update to the most current version from in the incubator now. I believe all projects are already updated to use oslo.rootwrap. I think the keystone guys are running into a similar issue where they want to remove a bunch of now-dead messaging code from keystone but can't because there are still some things in oslo-incubator using oslo.messaging code, or something weird like that. So maybe those modules are considered out of scope for this effort until the o-r/o-m code is completely out of o-i? There's a notifier middleware that uses the RPC code from the incubator still. I believe work on moving that module into
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
As a side to this, as an exercise I tried a oslo sync in cinder to see what kind of issues would arise and here are my findings so far: https://review.openstack.org/#/c/74786/ On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: On 2/19/2014 7:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Well I'll get the ball rolling... In the past when this has come up there is always a debate over should be just sync to sync because we should always be up to date, or is that dangerous and we should only sync when there is a need (which is what the review guidelines say now [1]). There are pros and cons: pros: - we get bug fixes that we didn't know existed - it should be less painful to sync if we do it more often cons: - it's more review overhead and some crazy guy thinks we need a special team dedicated to reviewing those changes :) - there are some changes in o-i that would break nova; I'm specifically thinking of the oslo RequestContext which has domain support now (or some other keystone thingy) and nova has it's own RequestContext - so if we did sync that from o-i it would change nova's logging context and break on us since we didn't use oslo context. For that last con, I'd argue that we should move to the oslo RequestContext, I'm not sure why we aren't. Would that module then not fall under low-hanging-fruit? I am classifying low hanging fruit as anything that doesn't require any nova changes to work. I think the DB API modules have been a concern for auto-syncing before too but I can't remember why now...something about possibly changing the behavior of how the nova migrations would work? But if they are already using the common code, I don't see the issue. AFAIK there is already a team working on db api syncing, so I was thinking of let them deal with it. This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? I think the keystone guys are running into a similar issue where they want to remove a bunch of now-dead messaging code from keystone but can't because there are still some things in oslo-incubator using oslo.messaging code, or something weird like that. So maybe those modules are considered out of scope for this effort until the o-r/o-m code is completely out of o-i? Finally, just like we'd like to have cores for each virt driver in nova and the neutron API in nova, I think this kind of thing, at least initially, would benefit from having some oslo cores involved in a team that are also familiar to a degree with nova, e.g. bnemec or dims. [1] https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist [2] https://review.openstack.org/#/c/73340/ -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon joe.gord...@gmail.com wrote: As a side to this, as an exercise I tried a oslo sync in cinder to see what kind of issues would arise and here are my findings so far: https://review.openstack.org/#/c/74786/ On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: On 2/19/2014 7:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Well I'll get the ball rolling... In the past when this has come up there is always a debate over should be just sync to sync because we should always be up to date, or is that dangerous and we should only sync when there is a need (which is what the review guidelines say now [1]). There are pros and cons: pros: - we get bug fixes that we didn't know existed - it should be less painful to sync if we do it more often cons: - it's more review overhead and some crazy guy thinks we need a special team dedicated to reviewing those changes :) - there are some changes in o-i that would break nova; I'm specifically thinking of the oslo RequestContext which has domain support now (or some other keystone thingy) and nova has it's own RequestContext - so if we did sync that from o-i it would change nova's logging context and break on us since we didn't use oslo context. For that last con, I'd argue that we should move to the oslo RequestContext, I'm not sure why we aren't. Would that module then not fall under low-hanging-fruit? I am classifying low hanging fruit as anything that doesn't require any nova changes to work. +1 I think the DB API modules have been a concern for auto-syncing before too but I can't remember why now...something about possibly changing the behavior of how the nova migrations would work? But if they are already using the common code, I don't see the issue. AFAIK there is already a team working on db api syncing, so I was thinking of let them deal with it. +1 Doug This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? I think the keystone guys are running into a similar issue where they want to remove a bunch of now-dead messaging code from keystone but can't because there are still some things in oslo-incubator using oslo.messaging code, or something weird like that. So maybe those modules are considered out of scope for this effort until the o-r/o-m code is completely out of o-i? Finally, just like we'd like to have cores for each virt driver in nova and the neutron API in nova, I think this kind of thing, at least initially, would benefit from having some oslo cores involved in a team that are also familiar to a degree with nova, e.g. bnemec or dims. [1] https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist [2] https://review.openstack.org/#/c/73340/ -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow
Shed a little bit of light on Matt's comment about Keystone removing oslo-incubator code and the issues we hit. Comments below. Best Regards, Lance Bragstad ldbra...@us.ibm.com Doug Hellmann doug.hellm...@dreamhost.com wrote on 02/19/2014 09:00:29 PM: From: Doug Hellmann doug.hellm...@dreamhost.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 02/19/2014 09:12 PM Subject: Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon joe.gord...@gmail.com wrote: As a side to this, as an exercise I tried a oslo sync in cinder to see what kind of issues would arise and here are my findings so far: https://review.openstack.org/#/c/74786/ On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: On 2/19/2014 7:13 PM, Joe Gordon wrote: Hi All, As many of you know most oslo-incubator code is wildly out of sync. Assuming we consider it a good idea to sync up oslo-incubator code before cutting Icehouse, then we have a problem. Today oslo-incubator code is synced in ad-hoc manor, resulting in duplicated efforts and wildly out of date code. Part of the challenges today are backwards incompatible changes and new oslo bugs. I expect that once we get a single project to have an up to date oslo-incubator copy it will make syncing a second project significantly easier. So because I (hopefully) have some karma built up in nova, I would like to volunteer nova to be the guinea pig. To fix this I would like to propose starting an oslo-incubator/nova sync team. They would be responsible for getting nova's oslo code up to date. I expect this work to involve: * Reviewing lots of oslo sync patches * Tracking the current sync patches * Syncing over the low hanging fruit, modules that work without changing nova. * Reporting bugs to oslo team * Working with oslo team to figure out how to deal with backwards incompatible changes * Update nova code or make oslo module backwards compatible * Track all this * Create a roadmap for other projects to follow (re: documentation) I am looking for volunteers to help with this effort, any takers? best, Joe Gordon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Well I'll get the ball rolling... In the past when this has come up there is always a debate over should be just sync to sync because we should always be up to date, or is that dangerous and we should only sync when there is a need (which is what the review guidelines say now [1]). There are pros and cons: pros: - we get bug fixes that we didn't know existed - it should be less painful to sync if we do it more often cons: - it's more review overhead and some crazy guy thinks we need a special team dedicated to reviewing those changes :) - there are some changes in o-i that would break nova; I'm specifically thinking of the oslo RequestContext which has domain support now (or some other keystone thingy) and nova has it's own RequestContext - so if we did sync that from o-i it would change nova's logging context and break on us since we didn't use oslo context. For that last con, I'd argue that we should move to the oslo RequestContext, I'm not sure why we aren't. Would that module then not fall under low-hanging-fruit? I am classifying low hanging fruit as anything that doesn't require any nova changes to work. +1 I think the DB API modules have been a concern for auto-syncing before too but I can't remember why now...something about possibly changing the behavior of how the nova migrations would work? But if they are already using the common code, I don't see the issue. AFAIK there is already a team working on db api syncing, so I was thinking of let them deal with it. +1 Doug This is kind of an aside, but I'm kind of confused now about how the syncs work with things that fall under oslo.rootwrap or oslo.messaging, like this patch [2]. It doesn't completely match the o-i patch, i.e. it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's in oslo.rootwrap now? But then why does the code still exist in oslo-incubator? I think the keystone guys are running into a similar issue where they want to remove a bunch of now-dead messaging code from keystone but can't because there are still some things in oslo-incubator using oslo.messaging code, or something weird like that. So maybe those modules are considered out of scope for this effort until the o-r/o-m code is completely out of o-i? For the Keystone work specifically, we were looking to remove the openstack.common.notifier and openstack.common.rpc modules from Keystone common
