Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-28 Thread Joe Gordon
Lets use https://etherpad.openstack.org/p/Icehouse-nova-oslo-sync to
keep track of things.

On Wed, Feb 26, 2014 at 5:10 PM, Joe Gordon joe.gord...@gmail.com wrote:
 GCB, Ben,

 Thanks for volunteering to help.

 GCB, reminded me that we should be doing this for python-novaclient in
 addition to nova itself. So that being said, as I see it here are the
 steps moving forward:

 Note: as previously mentionedin this thread there already is a team
 working on syncing oslo.db so we can ignore that for now (and once its
 ready they will propose patches, so we just have to do reviews).

 1) Review all outstanding nova/python-novaclient sync patches.
   https://review.openstack.org/#/c/72596/
   https://review.openstack.org/#/c/74560/
   https://review.openstack.org/#/c/75644/
 2) Using update.sh sync all low hanging fruit in both repos all at
 once. Low hanging fruit is anything that doesn't require a change
 outside of */openstack/common. As usual when doing these syncs we
 should list all patches being synced across, as well as document which
 modules we aren't syncing accross
./update.sh --base novaclient --config-file
 ../python-novaclient/openstack-common.conf --dest-dir
 ../python-novaclient/
 https://review.openstack.org/#/c/76642/
   ./update.sh --base nova --config-file ../nova/openstack-common.conf
 --dest-dir ../nova/
 3) At this point we should have a list of modules that are non-trivial
 to sync, now we can triage them and decide if they are oslo bugs or if
 nova/python-novaclient code needs updating.


 So for now we need reviews on the patches listed in 1, and someone to
 work on the low hanging fruit sync for nova. Followed by triaging of
 the non-low hanging fruit.

 Once we have the low hanging fruit out of the way lets sync up about
 how to handle the rest.

 best,
 Joe


 On Fri, Feb 21, 2014 at 6:26 PM, ChangBo Guo glongw...@gmail.com wrote:



 2014-02-22 5:09 GMT+08:00 Ben Nemec openst...@nemebean.com:

 /me finally catches up on -dev list traffic...

 On 2014-02-19 20:27, Doug Hellmann wrote:




 On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote:

 Hi All,

 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.

 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.


 Thank you for volunteering to spear-head this, Joe.

 +1

 To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date.  I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
   * Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)

 I am looking for volunteers to help with this effort, any takers?


 I will help, especially with reviews and tracking.

 I'm happy to help as well.  I always try to help with oslo syncs any time
 I'm made aware of problems anyway.

 What is our first step here?  Get the low-hanging fruit syncs proposed all
 at once?  Do them individually (taking into consideration the module deps,
 of course)?  If we're going to try to get this done for Icehouse then we
 probably need to start ASAP.

 -Ben

  I also would like to be volunteer of the new team :)
  -gcb


 --
 ChangBo Guo(gcb)

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-26 Thread Joe Gordon
GCB, Ben,

Thanks for volunteering to help.

GCB, reminded me that we should be doing this for python-novaclient in
addition to nova itself. So that being said, as I see it here are the
steps moving forward:

Note: as previously mentionedin this thread there already is a team
working on syncing oslo.db so we can ignore that for now (and once its
ready they will propose patches, so we just have to do reviews).

1) Review all outstanding nova/python-novaclient sync patches.
  https://review.openstack.org/#/c/72596/
  https://review.openstack.org/#/c/74560/
  https://review.openstack.org/#/c/75644/
2) Using update.sh sync all low hanging fruit in both repos all at
once. Low hanging fruit is anything that doesn't require a change
outside of */openstack/common. As usual when doing these syncs we
should list all patches being synced across, as well as document which
modules we aren't syncing accross
   ./update.sh --base novaclient --config-file
../python-novaclient/openstack-common.conf --dest-dir
../python-novaclient/
https://review.openstack.org/#/c/76642/
  ./update.sh --base nova --config-file ../nova/openstack-common.conf
--dest-dir ../nova/
3) At this point we should have a list of modules that are non-trivial
to sync, now we can triage them and decide if they are oslo bugs or if
nova/python-novaclient code needs updating.


So for now we need reviews on the patches listed in 1, and someone to
work on the low hanging fruit sync for nova. Followed by triaging of
the non-low hanging fruit.

Once we have the low hanging fruit out of the way lets sync up about
how to handle the rest.

best,
Joe


On Fri, Feb 21, 2014 at 6:26 PM, ChangBo Guo glongw...@gmail.com wrote:



 2014-02-22 5:09 GMT+08:00 Ben Nemec openst...@nemebean.com:

 /me finally catches up on -dev list traffic...

 On 2014-02-19 20:27, Doug Hellmann wrote:




 On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote:

 Hi All,

 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.

 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.


 Thank you for volunteering to spear-head this, Joe.

 +1

 To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date.  I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
   * Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)

 I am looking for volunteers to help with this effort, any takers?


 I will help, especially with reviews and tracking.

 I'm happy to help as well.  I always try to help with oslo syncs any time
 I'm made aware of problems anyway.

 What is our first step here?  Get the low-hanging fruit syncs proposed all
 at once?  Do them individually (taking into consideration the module deps,
 of course)?  If we're going to try to get this done for Icehouse then we
 probably need to start ASAP.

 -Ben

  I also would like to be volunteer of the new team :)
  -gcb


 --
 ChangBo Guo(gcb)

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-21 Thread Ben Nemec
 

/me finally catches up on -dev list traffic... 

On 2014-02-19 20:27, Doug Hellmann wrote: 

 On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote:
 
 Hi All,
 
 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.
 
 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.
 
 Thank you for volunteering to spear-head this, Joe.

+1 

 To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date. I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing 
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
 * Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)
 
 I am looking for volunteers to help with this effort, any takers?
 
 I will help, especially with reviews and tracking.

I'm happy to help as well. I always try to help with oslo syncs any time
I'm made aware of problems anyway. 

What is our first step here? Get the low-hanging fruit syncs proposed
all at once? Do them individually (taking into consideration the module
deps, of course)? If we're going to try to get this done for Icehouse
then we probably need to start ASAP. 

-Ben 

 We are going to want someone from the team working on the db modules to 
 participate as well, since we know that's one area where the API has diverged 
 some (although we did take backwards compatibility into account). Victor, can 
 you help find us a volunteer? 
 
 Doug 
 
 best,
 Joe Gordon
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev [1]
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  [1]

 

Links:
--
[1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-21 Thread ChangBo Guo
2014-02-22 5:09 GMT+08:00 Ben Nemec openst...@nemebean.com:

  /me finally catches up on -dev list traffic...

 On 2014-02-19 20:27, Doug Hellmann wrote:




 On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote:

 Hi All,

 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.

 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.


  Thank you for volunteering to spear-head this, Joe.

   +1

   To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date.  I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
   * Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)

 I am looking for volunteers to help with this effort, any takers?


  I will help, especially with reviews and tracking.

   I'm happy to help as well.  I always try to help with oslo syncs any
 time I'm made aware of problems anyway.

 What is our first step here?  Get the low-hanging fruit syncs proposed all
 at once?  Do them individually (taking into consideration the module deps,
 of course)?  If we're going to try to get this done for Icehouse then we
 probably need to start ASAP.

 -Ben

 I also would like to be volunteer of the new team :)
 -gcb


-- 
ChangBo Guo(gcb)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-20 Thread Roman Podoliaka
Hi all,

I'm ready to help with syncing of DB code. But we'll need reviewers
attention in both oslo-incubator in nova :)

Thanks,
Roman

On Thu, Feb 20, 2014 at 5:37 AM, Lance D Bragstad ldbra...@us.ibm.com wrote:
 Shed a little bit of light on Matt's comment about Keystone removing
 oslo-incubator code and the issues we hit. Comments below.


 Best Regards,

 Lance Bragstad
 ldbra...@us.ibm.com

 Doug Hellmann doug.hellm...@dreamhost.com wrote on 02/19/2014 09:00:29 PM:

 From: Doug Hellmann doug.hellm...@dreamhost.com
 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org,
 Date: 02/19/2014 09:12 PM
 Subject: Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator
 sync workflow





 On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon joe.gord...@gmail.com wrote:
 As a side to this, as an exercise I tried a oslo sync in cinder to see
 what kind of issues would arise and here are my findings so far:
 https://review.openstack.org/#/c/74786/

 On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann
 mrie...@linux.vnet.ibm.com wrote:
 
 
  On 2/19/2014 7:13 PM, Joe Gordon wrote:
 
  Hi All,
 
  As many of you know most oslo-incubator code is wildly out of sync.
  Assuming we consider it a good idea to sync up oslo-incubator code
  before cutting Icehouse, then we have a problem.
 
  Today oslo-incubator code is synced in ad-hoc manor, resulting in
  duplicated efforts and wildly out of date code. Part of the challenges
  today are backwards incompatible changes and new oslo bugs. I expect
  that once we get a single project to have an up to date oslo-incubator
  copy it will make syncing a second project significantly easier. So
  because I (hopefully) have some karma built up in nova, I would like
  to volunteer nova to be the guinea pig.
 
 
  To fix this I would like to propose starting an oslo-incubator/nova
  sync team. They would be responsible for getting nova's oslo code up
  to date.  I expect this work to involve:
  * Reviewing lots of oslo sync patches
  * Tracking the current sync patches
  * Syncing over the low hanging fruit, modules that work without
  changing
  nova.
  * Reporting bugs to oslo team
  * Working with oslo team to figure out how to deal with backwards
  incompatible changes
 * Update nova code or make oslo module backwards compatible
  * Track all this
  * Create a roadmap for other projects to follow (re: documentation)
 
  I am looking for volunteers to help with this effort, any takers?
 
 
  best,
  Joe Gordon
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
  Well I'll get the ball rolling...
 
  In the past when this has come up there is always a debate over should
  be
  just sync to sync because we should always be up to date, or is that
  dangerous and we should only sync when there is a need (which is what
  the
  review guidelines say now [1]).  There are pros and cons:
 
  pros:
 
  - we get bug fixes that we didn't know existed
  - it should be less painful to sync if we do it more often
 
  cons:
 
  - it's more review overhead and some crazy guy thinks we need a special
  team
  dedicated to reviewing those changes :)
  - there are some changes in o-i that would break nova; I'm specifically
  thinking of the oslo RequestContext which has domain support now (or
  some
  other keystone thingy) and nova has it's own RequestContext - so if we
  did
  sync that from o-i it would change nova's logging context and break on
  us
  since we didn't use oslo context.
 
  For that last con, I'd argue that we should move to the oslo
  RequestContext,
  I'm not sure why we aren't.  Would that module then not fall under
  low-hanging-fruit?

 I am classifying low hanging fruit as anything that doesn't require
 any nova changes to work.

 +1
  I think the DB API modules have been a concern for auto-syncing before
  too
  but I can't remember why now...something about possibly changing the
  behavior of how the nova migrations would work?  But if they are already
  using the common code, I don't see the issue.

 AFAIK there is already a team working on db api syncing, so I was
 thinking of let them deal with it.

 +1

 Doug

 
  This is kind of an aside, but I'm kind of confused now about how the
  syncs
  work with things that fall under oslo.rootwrap or oslo.messaging, like
  this
  patch [2].  It doesn't completely match the o-i patch, i.e. it's not
  syncing
  over openstack/common/rootwrap/wrapper.py, and I'm assuming because
  that's
  in oslo.rootwrap now?  But then why does the code still exist in
  oslo-incubator?
 
  I think the keystone guys are running into a similar issue where they
  want
  to remove a bunch of now-dead messaging code from keystone but can't
  because
  there are still some things in oslo-incubator using oslo.messaging code,
  or
  something weird like that. So maybe those

Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-20 Thread Victor Sergeyev
Hello All

I and Roman Podoliaka are familiar with the changes made to common db code,
so we are ready to help with syncing it to OS projects.
But we want to ask you for more activity in reviewing of these patches.

Thanks, Victor


On Thu, Feb 20, 2014 at 4:27 AM, Doug Hellmann
doug.hellm...@dreamhost.comwrote:




 On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote:

 Hi All,

 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.

 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.


 Thank you for volunteering to spear-head this, Joe.


 To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date.  I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
   * Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)

 I am looking for volunteers to help with this effort, any takers?


 I will help, especially with reviews and tracking.

 We are going to want someone from the team working on the db modules to
 participate as well, since we know that's one area where the API has
 diverged some (although we did take backwards compatibility into account).
 Victor, can you help find us a volunteer?

 Doug





 best,
 Joe Gordon

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-20 Thread Thierry Carrez
Matt Riedemann wrote:
 This is kind of an aside, but I'm kind of confused now about how the
 syncs work with things that fall under oslo.rootwrap or oslo.messaging,
 like this patch [2].  It doesn't completely match the o-i patch, i.e.
 it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm
 assuming because that's in oslo.rootwrap now?  But then why does the
 code still exist in oslo-incubator?

FWIW the code was recently removed from the oslo-incubator, once Neutron
(the last of the rootwrap-consuming projects) got migrated to using
oslo.rootwrap.

 [2] https://review.openstack.org/#/c/73340/

This one syncs changes from https://review.openstack.org/#/c/63094

63094 should never have been approved, since rootwrap in oslo-incubator
was frozen (graduating). Now the changes are lost, since they were
never proposed to oslo.rootwrap, and the code in the incubator was
cleaned up.

I'll comment on the 73340 review to try to solve this mess.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-19 Thread Joe Gordon
Hi All,

As many of you know most oslo-incubator code is wildly out of sync.
Assuming we consider it a good idea to sync up oslo-incubator code
before cutting Icehouse, then we have a problem.

Today oslo-incubator code is synced in ad-hoc manor, resulting in
duplicated efforts and wildly out of date code. Part of the challenges
today are backwards incompatible changes and new oslo bugs. I expect
that once we get a single project to have an up to date oslo-incubator
copy it will make syncing a second project significantly easier. So
because I (hopefully) have some karma built up in nova, I would like
to volunteer nova to be the guinea pig.


To fix this I would like to propose starting an oslo-incubator/nova
sync team. They would be responsible for getting nova's oslo code up
to date.  I expect this work to involve:
* Reviewing lots of oslo sync patches
* Tracking the current sync patches
* Syncing over the low hanging fruit, modules that work without changing nova.
* Reporting bugs to oslo team
* Working with oslo team to figure out how to deal with backwards
incompatible changes
  * Update nova code or make oslo module backwards compatible
* Track all this
* Create a roadmap for other projects to follow (re: documentation)

I am looking for volunteers to help with this effort, any takers?


best,
Joe Gordon

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-19 Thread Matt Riedemann



On 2/19/2014 7:13 PM, Joe Gordon wrote:

Hi All,

As many of you know most oslo-incubator code is wildly out of sync.
Assuming we consider it a good idea to sync up oslo-incubator code
before cutting Icehouse, then we have a problem.

Today oslo-incubator code is synced in ad-hoc manor, resulting in
duplicated efforts and wildly out of date code. Part of the challenges
today are backwards incompatible changes and new oslo bugs. I expect
that once we get a single project to have an up to date oslo-incubator
copy it will make syncing a second project significantly easier. So
because I (hopefully) have some karma built up in nova, I would like
to volunteer nova to be the guinea pig.


To fix this I would like to propose starting an oslo-incubator/nova
sync team. They would be responsible for getting nova's oslo code up
to date.  I expect this work to involve:
* Reviewing lots of oslo sync patches
* Tracking the current sync patches
* Syncing over the low hanging fruit, modules that work without changing nova.
* Reporting bugs to oslo team
* Working with oslo team to figure out how to deal with backwards
incompatible changes
   * Update nova code or make oslo module backwards compatible
* Track all this
* Create a roadmap for other projects to follow (re: documentation)

I am looking for volunteers to help with this effort, any takers?


best,
Joe Gordon

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Well I'll get the ball rolling...

In the past when this has come up there is always a debate over should 
be just sync to sync because we should always be up to date, or is that 
dangerous and we should only sync when there is a need (which is what 
the review guidelines say now [1]).  There are pros and cons:


pros:

- we get bug fixes that we didn't know existed
- it should be less painful to sync if we do it more often

cons:

- it's more review overhead and some crazy guy thinks we need a special 
team dedicated to reviewing those changes :)
- there are some changes in o-i that would break nova; I'm specifically 
thinking of the oslo RequestContext which has domain support now (or 
some other keystone thingy) and nova has it's own RequestContext - so if 
we did sync that from o-i it would change nova's logging context and 
break on us since we didn't use oslo context.


For that last con, I'd argue that we should move to the oslo 
RequestContext, I'm not sure why we aren't.  Would that module then not 
fall under low-hanging-fruit?


I think the DB API modules have been a concern for auto-syncing before 
too but I can't remember why now...something about possibly changing the 
behavior of how the nova migrations would work?  But if they are already 
using the common code, I don't see the issue.


This is kind of an aside, but I'm kind of confused now about how the 
syncs work with things that fall under oslo.rootwrap or oslo.messaging, 
like this patch [2].  It doesn't completely match the o-i patch, i.e. 
it's not syncing over openstack/common/rootwrap/wrapper.py, and I'm 
assuming because that's in oslo.rootwrap now?  But then why does the 
code still exist in oslo-incubator?


I think the keystone guys are running into a similar issue where they 
want to remove a bunch of now-dead messaging code from keystone but 
can't because there are still some things in oslo-incubator using 
oslo.messaging code, or something weird like that. So maybe those 
modules are considered out of scope for this effort until the o-r/o-m 
code is completely out of o-i?


Finally, just like we'd like to have cores for each virt driver in nova 
and the neutron API in nova, I think this kind of thing, at least 
initially, would benefit from having some oslo cores involved in a team 
that are also familiar to a degree with nova, e.g. bnemec or dims.


[1] https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist
[2] https://review.openstack.org/#/c/73340/

--

Thanks,

Matt Riedemann


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-19 Thread Doug Hellmann
On Wed, Feb 19, 2014 at 8:13 PM, Joe Gordon joe.gord...@gmail.com wrote:

 Hi All,

 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.

 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.


Thank you for volunteering to spear-head this, Joe. 


 To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date.  I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
   * Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)

 I am looking for volunteers to help with this effort, any takers?


I will help, especially with reviews and tracking.

We are going to want someone from the team working on the db modules to
participate as well, since we know that's one area where the API has
diverged some (although we did take backwards compatibility into account).
Victor, can you help find us a volunteer?

Doug





 best,
 Joe Gordon

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-19 Thread Doug Hellmann
On Wed, Feb 19, 2014 at 9:20 PM, Matt Riedemann
mrie...@linux.vnet.ibm.comwrote:



 On 2/19/2014 7:13 PM, Joe Gordon wrote:

 Hi All,

 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.

 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.


 To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date.  I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
* Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)

 I am looking for volunteers to help with this effort, any takers?


 best,
 Joe Gordon

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 Well I'll get the ball rolling...

 In the past when this has come up there is always a debate over should be
 just sync to sync because we should always be up to date, or is that
 dangerous and we should only sync when there is a need (which is what the
 review guidelines say now [1]).  There are pros and cons:

 pros:

 - we get bug fixes that we didn't know existed
 - it should be less painful to sync if we do it more often

 cons:

 - it's more review overhead and some crazy guy thinks we need a special
 team dedicated to reviewing those changes :)
 - there are some changes in o-i that would break nova; I'm specifically
 thinking of the oslo RequestContext which has domain support now (or some
 other keystone thingy) and nova has it's own RequestContext - so if we did
 sync that from o-i it would change nova's logging context and break on us
 since we didn't use oslo context.


Another con is that if we do find a critical bug in an incubator module,
and a project that uses that module is far out of date, applying the fix
may be more difficult. (This is also another motivation for moving code
out of the incubator entirely, but as Joe pointed out earlier today, that's
not really a short-term solution.)



 For that last con, I'd argue that we should move to the oslo
 RequestContext, I'm not sure why we aren't.  Would that module then not
 fall under low-hanging-fruit?

 I think the DB API modules have been a concern for auto-syncing before too
 but I can't remember why now...something about possibly changing the
 behavior of how the nova migrations would work?  But if they are already
 using the common code, I don't see the issue.


There has been some recent work on the db code to make it more suitable for
use in some of the other projects that don't have a single global session
pool. There's a compatibility shim, which should make the update painless,
but it's not just a simple file copy.



 This is kind of an aside, but I'm kind of confused now about how the syncs
 work with things that fall under oslo.rootwrap or oslo.messaging, like this
 patch [2].  It doesn't completely match the o-i patch, i.e. it's not
 syncing over openstack/common/rootwrap/wrapper.py, and I'm assuming
 because that's in oslo.rootwrap now?  But then why does the code still
 exist in oslo-incubator?


After a module graduates to a library, we treat the incubator copy as the
stable branch until all of the integrated projects that consume the
module have migrated to the new library. That way if bugs are found, the
fixes can be applied to a project without having to also migrate to the
library.

So, the best action is to port to the library. As a fall back, at least
update to the most current version from in the incubator now. I believe all
projects are already updated to use oslo.rootwrap.

I think the keystone guys are running into a similar issue where they want
 to remove a bunch of now-dead messaging code from keystone but can't
 because there are still some things in oslo-incubator using oslo.messaging
 code, or something weird like that. So maybe those modules are considered
 out of scope for this effort until the o-r/o-m code is completely out of
 o-i?


There's a notifier middleware that uses the RPC code from the incubator
still. I believe work on moving that module into 

Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-19 Thread Joe Gordon
As a side to this, as an exercise I tried a oslo sync in cinder to see
what kind of issues would arise and here are my findings so far:
https://review.openstack.org/#/c/74786/

On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann
mrie...@linux.vnet.ibm.com wrote:


 On 2/19/2014 7:13 PM, Joe Gordon wrote:

 Hi All,

 As many of you know most oslo-incubator code is wildly out of sync.
 Assuming we consider it a good idea to sync up oslo-incubator code
 before cutting Icehouse, then we have a problem.

 Today oslo-incubator code is synced in ad-hoc manor, resulting in
 duplicated efforts and wildly out of date code. Part of the challenges
 today are backwards incompatible changes and new oslo bugs. I expect
 that once we get a single project to have an up to date oslo-incubator
 copy it will make syncing a second project significantly easier. So
 because I (hopefully) have some karma built up in nova, I would like
 to volunteer nova to be the guinea pig.


 To fix this I would like to propose starting an oslo-incubator/nova
 sync team. They would be responsible for getting nova's oslo code up
 to date.  I expect this work to involve:
 * Reviewing lots of oslo sync patches
 * Tracking the current sync patches
 * Syncing over the low hanging fruit, modules that work without changing
 nova.
 * Reporting bugs to oslo team
 * Working with oslo team to figure out how to deal with backwards
 incompatible changes
* Update nova code or make oslo module backwards compatible
 * Track all this
 * Create a roadmap for other projects to follow (re: documentation)

 I am looking for volunteers to help with this effort, any takers?


 best,
 Joe Gordon

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 Well I'll get the ball rolling...

 In the past when this has come up there is always a debate over should be
 just sync to sync because we should always be up to date, or is that
 dangerous and we should only sync when there is a need (which is what the
 review guidelines say now [1]).  There are pros and cons:

 pros:

 - we get bug fixes that we didn't know existed
 - it should be less painful to sync if we do it more often

 cons:

 - it's more review overhead and some crazy guy thinks we need a special team
 dedicated to reviewing those changes :)
 - there are some changes in o-i that would break nova; I'm specifically
 thinking of the oslo RequestContext which has domain support now (or some
 other keystone thingy) and nova has it's own RequestContext - so if we did
 sync that from o-i it would change nova's logging context and break on us
 since we didn't use oslo context.

 For that last con, I'd argue that we should move to the oslo RequestContext,
 I'm not sure why we aren't.  Would that module then not fall under
 low-hanging-fruit?

I am classifying low hanging fruit as anything that doesn't require
any nova changes to work.


 I think the DB API modules have been a concern for auto-syncing before too
 but I can't remember why now...something about possibly changing the
 behavior of how the nova migrations would work?  But if they are already
 using the common code, I don't see the issue.

AFAIK there is already a team working on db api syncing, so I was
thinking of let them deal with it.


 This is kind of an aside, but I'm kind of confused now about how the syncs
 work with things that fall under oslo.rootwrap or oslo.messaging, like this
 patch [2].  It doesn't completely match the o-i patch, i.e. it's not syncing
 over openstack/common/rootwrap/wrapper.py, and I'm assuming because that's
 in oslo.rootwrap now?  But then why does the code still exist in
 oslo-incubator?

 I think the keystone guys are running into a similar issue where they want
 to remove a bunch of now-dead messaging code from keystone but can't because
 there are still some things in oslo-incubator using oslo.messaging code, or
 something weird like that. So maybe those modules are considered out of
 scope for this effort until the o-r/o-m code is completely out of o-i?

 Finally, just like we'd like to have cores for each virt driver in nova and
 the neutron API in nova, I think this kind of thing, at least initially,
 would benefit from having some oslo cores involved in a team that are also
 familiar to a degree with nova, e.g. bnemec or dims.

 [1] https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist
 [2] https://review.openstack.org/#/c/73340/

 --

 Thanks,

 Matt Riedemann


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-19 Thread Doug Hellmann
On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon joe.gord...@gmail.com wrote:

 As a side to this, as an exercise I tried a oslo sync in cinder to see
 what kind of issues would arise and here are my findings so far:
 https://review.openstack.org/#/c/74786/

 On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann
 mrie...@linux.vnet.ibm.com wrote:
 
 
  On 2/19/2014 7:13 PM, Joe Gordon wrote:
 
  Hi All,
 
  As many of you know most oslo-incubator code is wildly out of sync.
  Assuming we consider it a good idea to sync up oslo-incubator code
  before cutting Icehouse, then we have a problem.
 
  Today oslo-incubator code is synced in ad-hoc manor, resulting in
  duplicated efforts and wildly out of date code. Part of the challenges
  today are backwards incompatible changes and new oslo bugs. I expect
  that once we get a single project to have an up to date oslo-incubator
  copy it will make syncing a second project significantly easier. So
  because I (hopefully) have some karma built up in nova, I would like
  to volunteer nova to be the guinea pig.
 
 
  To fix this I would like to propose starting an oslo-incubator/nova
  sync team. They would be responsible for getting nova's oslo code up
  to date.  I expect this work to involve:
  * Reviewing lots of oslo sync patches
  * Tracking the current sync patches
  * Syncing over the low hanging fruit, modules that work without changing
  nova.
  * Reporting bugs to oslo team
  * Working with oslo team to figure out how to deal with backwards
  incompatible changes
 * Update nova code or make oslo module backwards compatible
  * Track all this
  * Create a roadmap for other projects to follow (re: documentation)
 
  I am looking for volunteers to help with this effort, any takers?
 
 
  best,
  Joe Gordon
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
  Well I'll get the ball rolling...
 
  In the past when this has come up there is always a debate over should be
  just sync to sync because we should always be up to date, or is that
  dangerous and we should only sync when there is a need (which is what the
  review guidelines say now [1]).  There are pros and cons:
 
  pros:
 
  - we get bug fixes that we didn't know existed
  - it should be less painful to sync if we do it more often
 
  cons:
 
  - it's more review overhead and some crazy guy thinks we need a special
 team
  dedicated to reviewing those changes :)
  - there are some changes in o-i that would break nova; I'm specifically
  thinking of the oslo RequestContext which has domain support now (or some
  other keystone thingy) and nova has it's own RequestContext - so if we
 did
  sync that from o-i it would change nova's logging context and break on us
  since we didn't use oslo context.
 
  For that last con, I'd argue that we should move to the oslo
 RequestContext,
  I'm not sure why we aren't.  Would that module then not fall under
  low-hanging-fruit?

 I am classifying low hanging fruit as anything that doesn't require
 any nova changes to work.


+1


  I think the DB API modules have been a concern for auto-syncing before
 too
  but I can't remember why now...something about possibly changing the
  behavior of how the nova migrations would work?  But if they are already
  using the common code, I don't see the issue.

 AFAIK there is already a team working on db api syncing, so I was
 thinking of let them deal with it.


+1

Doug



 
  This is kind of an aside, but I'm kind of confused now about how the
 syncs
  work with things that fall under oslo.rootwrap or oslo.messaging, like
 this
  patch [2].  It doesn't completely match the o-i patch, i.e. it's not
 syncing
  over openstack/common/rootwrap/wrapper.py, and I'm assuming because
 that's
  in oslo.rootwrap now?  But then why does the code still exist in
  oslo-incubator?
 
  I think the keystone guys are running into a similar issue where they
 want
  to remove a bunch of now-dead messaging code from keystone but can't
 because
  there are still some things in oslo-incubator using oslo.messaging code,
 or
  something weird like that. So maybe those modules are considered out of
  scope for this effort until the o-r/o-m code is completely out of o-i?
 
  Finally, just like we'd like to have cores for each virt driver in nova
 and
  the neutron API in nova, I think this kind of thing, at least initially,
  would benefit from having some oslo cores involved in a team that are
 also
  familiar to a degree with nova, e.g. bnemec or dims.
 
  [1]
 https://wiki.openstack.org/wiki/ReviewChecklist#Oslo_Syncing_Checklist
  [2] https://review.openstack.org/#/c/73340/
 
  --
 
  Thanks,
 
  Matt Riedemann
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 

Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator sync workflow

2014-02-19 Thread Lance D Bragstad

Shed a little bit of light on Matt's comment about Keystone removing
oslo-incubator code and the issues we hit. Comments below.


Best Regards,

Lance Bragstad
ldbra...@us.ibm.com

Doug Hellmann doug.hellm...@dreamhost.com wrote on 02/19/2014 09:00:29
PM:

 From: Doug Hellmann doug.hellm...@dreamhost.com
 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org,
 Date: 02/19/2014 09:12 PM
 Subject: Re: [openstack-dev] [nova][oslo] Changes to oslo-incubator
 sync workflow



 On Wed, Feb 19, 2014 at 9:52 PM, Joe Gordon joe.gord...@gmail.com
wrote:
 As a side to this, as an exercise I tried a oslo sync in cinder to see
 what kind of issues would arise and here are my findings so far:
 https://review.openstack.org/#/c/74786/

 On Wed, Feb 19, 2014 at 6:20 PM, Matt Riedemann
 mrie...@linux.vnet.ibm.com wrote:
 
 
  On 2/19/2014 7:13 PM, Joe Gordon wrote:
 
  Hi All,
 
  As many of you know most oslo-incubator code is wildly out of sync.
  Assuming we consider it a good idea to sync up oslo-incubator code
  before cutting Icehouse, then we have a problem.
 
  Today oslo-incubator code is synced in ad-hoc manor, resulting in
  duplicated efforts and wildly out of date code. Part of the challenges
  today are backwards incompatible changes and new oslo bugs. I expect
  that once we get a single project to have an up to date oslo-incubator
  copy it will make syncing a second project significantly easier. So
  because I (hopefully) have some karma built up in nova, I would like
  to volunteer nova to be the guinea pig.
 
 
  To fix this I would like to propose starting an oslo-incubator/nova
  sync team. They would be responsible for getting nova's oslo code up
  to date.  I expect this work to involve:
  * Reviewing lots of oslo sync patches
  * Tracking the current sync patches
  * Syncing over the low hanging fruit, modules that work without
changing
  nova.
  * Reporting bugs to oslo team
  * Working with oslo team to figure out how to deal with backwards
  incompatible changes
     * Update nova code or make oslo module backwards compatible
  * Track all this
  * Create a roadmap for other projects to follow (re: documentation)
 
  I am looking for volunteers to help with this effort, any takers?
 
 
  best,
  Joe Gordon
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
  Well I'll get the ball rolling...
 
  In the past when this has come up there is always a debate over should
be
  just sync to sync because we should always be up to date, or is that
  dangerous and we should only sync when there is a need (which is what
the
  review guidelines say now [1]).  There are pros and cons:
 
  pros:
 
  - we get bug fixes that we didn't know existed
  - it should be less painful to sync if we do it more often
 
  cons:
 
  - it's more review overhead and some crazy guy thinks we need a special
team
  dedicated to reviewing those changes :)
  - there are some changes in o-i that would break nova; I'm specifically
  thinking of the oslo RequestContext which has domain support now (or
some
  other keystone thingy) and nova has it's own RequestContext - so if we
did
  sync that from o-i it would change nova's logging context and break on
us
  since we didn't use oslo context.
 
  For that last con, I'd argue that we should move to the oslo
RequestContext,
  I'm not sure why we aren't.  Would that module then not fall under
  low-hanging-fruit?

 I am classifying low hanging fruit as anything that doesn't require
 any nova changes to work.

 +1
  I think the DB API modules have been a concern for auto-syncing before
too
  but I can't remember why now...something about possibly changing the
  behavior of how the nova migrations would work?  But if they are
already
  using the common code, I don't see the issue.

 AFAIK there is already a team working on db api syncing, so I was
 thinking of let them deal with it.

 +1

 Doug

 
  This is kind of an aside, but I'm kind of confused now about how the
syncs
  work with things that fall under oslo.rootwrap or oslo.messaging, like
this
  patch [2].  It doesn't completely match the o-i patch, i.e. it's not
syncing
  over openstack/common/rootwrap/wrapper.py, and I'm assuming because
that's
  in oslo.rootwrap now?  But then why does the code still exist in
  oslo-incubator?
 
  I think the keystone guys are running into a similar issue where they
want
  to remove a bunch of now-dead messaging code from keystone but can't
because
  there are still some things in oslo-incubator using oslo.messaging
code, or
  something weird like that. So maybe those modules are considered out of
  scope for this effort until the o-r/o-m code is completely out of o-i?
 

For the Keystone work specifically, we were looking to remove the
openstack.common.notifier
and openstack.common.rpc modules from Keystone common