Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 15/12/14 18:57, Doug Hellmann wrote:
 There may be a similar problem managing dependencies on libraries
 that live outside of either tree. I assume you already decided how
 to handle that. Are you doing the same thing, and adding the
 requirements to neutron’s lists?

I guess the idea is to keep in neutron-*aas only those oslo-incubator
modules that are used there solely (=not used in main repo).

I think requirements are a bit easier and should track all direct
dependencies in each of the repos, so that in case main repo decides
to drop one, neutron-*aas repos are not broken.

For requirements, it's different because there is no major burden due
to duplicate entries in repos.

 
 On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com
 wrote:
 
 Hi all,
 
 Ihar and I discussed this on IRC, and are going forward with
 option 2 unless someone has a big problem with it.
 
 Thanks, Doug
 
 
 On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com
 wrote:
 
 Hi Ihar,
 
 I’m actually in favor of option 2, but it implies a few things
 about your time, and I wanted to chat with you before
 presuming.
 
 Maintenance can not involve breaking changes. At this point,
 the co-gate will block it.  Also, oslo graduation changes will
 have to be made in the services repos first, and then Neutron.
 
 Thanks, doug
 
 
 On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com
 wrote:
 
 Hi all,
 
 the question arose recently in one of reviews for neutron-*aas
 repos to remove all oslo-incubator code from those repos since
 it's duplicated in neutron main repo. (You can find the link to the
 review at the end of the email.)
 
 Brief hostory: neutron repo was recently split into 4 pieces
 (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split
 resulted in each repository keeping their own copy of 
 neutron/openstack/common/... tree (currently unused in all 
 neutron-*aas repos that are still bound to modules from main
 repo).
 
 As a oslo liaison for the project, I wonder what's the best way to 
 manage oslo-incubator files. We have several options:
 
 1. just kill all the neutron/openstack/common/ trees from
 neutron-*aas repositories and continue using modules from main
 repo.
 
 2. kill all duplicate modules from neutron-*aas repos and leave
 only those that are used in those repos but not in main repo.
 
 3. fully duplicate all those modules in each of four repos that use
 them.
 
 I think option 1. is a straw man, since we should be able to
 introduce new oslo-incubator modules into neutron-*aas repos even
 if they are not used in main repo.
 
 Option 2. is good when it comes to synching non-breaking bug fixes
 (or security fixes) from oslo-incubator, in that it will require
 only one sync patch instead of e.g. four. At the same time there
 may be potential issues when synchronizing updates from
 oslo-incubator that would break API and hence require changes to
 each of the modules that use it. Since we don't support atomic
 merges for multiple projects in gate, we will need to be cautious
 about those updates, and we will still need to leave neutron-*aas
 repos broken for some time (though the time may be mitigated with
 care).
 
 Option 3. is vice versa - in theory, you get total decoupling,
 meaning no oslo-incubator updates in main repo are expected to
 break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
 I would vote for option 2., for two reasons: - most oslo-incubator
 syncs are non-breaking, and we may effectively apply care to
 updates that may result in potential breakage (f.e. being able to
 trigger an integrated run for each of neutron-*aas repos with the
 main sync patch, if there are any concerns). - it will make oslo
 liaison life a lot easier. OK, I'm probably too selfish on that.
 ;) - it will make stable maintainers life a lot easier. The main
 reason why stable maintainers and distributions like recent oslo
 graduation movement is that we don't need to track each bug fix we
 need in every project, and waste lots of cycles on it. Being able
 to fix a bug in one place only is *highly* anticipated. [OK, I'm
 quite selfish on that one too.] - it's a delusion that there will
 be no neutron-main syncs that will break neutron-*aas repos ever.
 There can still be problems due to incompatibility between neutron
 main and neutron-*aas code resulted EXACTLY because multiple parts
 of the same process use different versions of the same module.
 
 That said, Doug Wiegley (lbaas core) seems to be in favour of
 option 3. due to lower coupling that is achieved in that way. I
 know that lbaas team had a bad experience due to tight coupling to
 neutron project in the past, so I appreciate their concerns.
 
 All in all, we should come up with some standard solution for both 
 advanced services that are already split out, *and* upcoming
 vendor plugin shrinking initiative.
 
 The initial discussion is captured at: 
 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 15/12/14 17:22, Doug Wiegley wrote:
 Hi Ihar,
 
 I’m actually in favor of option 2, but it implies a few things
 about your time, and I wanted to chat with you before presuming.

I think split didn't mean moving project trees under separate
governance, so I assume oslo (doc, qa, ...) liaisons should not be
split either.

 
 Maintenance can not involve breaking changes. At this point, the
 co-gate will block it.  Also, oslo graduation changes will have to
 be made in the services repos first, and then Neutron.

Do you mean that a change to oslo-incubator modules is co-gated (not
just co-checked with no vote) with each of advanced services?

As I pointed in my previous email, sometimes breakages are inescapable.

Consider a change to neutron oslo-incubator module used commonly in
all repos that breaks API (they are quite rare, but still have a
chance of happening once in a while). If we would co-gate main neutron
repo changes with services, it will mean that we won't be able to
merge the change.

That would probably suggest that we go forward with option 3 and
manage all incubator files separately in each of the trees, though,
again, breakages are still possible in that scenario via introducing
incompatibility between versions of incubator modules in separate repos.

So we should be realistic about it and plan forward how we deal
potential breakages that *may* occur.

As for oslo library graduations, the order is not really significant.
What is significant is that we drop oslo-incubator module from main
neutron repo only after all other neutron-*aas repos migrate to
appropriate oslo.* library. The neutron migration itself may occur in
parallel (by postponing module drop later).

 
 Thanks, doug
 
 
 On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com
 wrote:
 
 Hi all,
 
 the question arose recently in one of reviews for neutron-*aas
 repos to remove all oslo-incubator code from those repos since
 it's duplicated in neutron main repo. (You can find the link to the
 review at the end of the email.)
 
 Brief hostory: neutron repo was recently split into 4 pieces
 (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split
 resulted in each repository keeping their own copy of 
 neutron/openstack/common/... tree (currently unused in all 
 neutron-*aas repos that are still bound to modules from main
 repo).
 
 As a oslo liaison for the project, I wonder what's the best way to 
 manage oslo-incubator files. We have several options:
 
 1. just kill all the neutron/openstack/common/ trees from
 neutron-*aas repositories and continue using modules from main
 repo.
 
 2. kill all duplicate modules from neutron-*aas repos and leave
 only those that are used in those repos but not in main repo.
 
 3. fully duplicate all those modules in each of four repos that use
 them.
 
 I think option 1. is a straw man, since we should be able to
 introduce new oslo-incubator modules into neutron-*aas repos even
 if they are not used in main repo.
 
 Option 2. is good when it comes to synching non-breaking bug fixes
 (or security fixes) from oslo-incubator, in that it will require
 only one sync patch instead of e.g. four. At the same time there
 may be potential issues when synchronizing updates from
 oslo-incubator that would break API and hence require changes to
 each of the modules that use it. Since we don't support atomic
 merges for multiple projects in gate, we will need to be cautious
 about those updates, and we will still need to leave neutron-*aas
 repos broken for some time (though the time may be mitigated with
 care).
 
 Option 3. is vice versa - in theory, you get total decoupling,
 meaning no oslo-incubator updates in main repo are expected to
 break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
 I would vote for option 2., for two reasons: - most oslo-incubator
 syncs are non-breaking, and we may effectively apply care to
 updates that may result in potential breakage (f.e. being able to
 trigger an integrated run for each of neutron-*aas repos with the
 main sync patch, if there are any concerns). - it will make oslo
 liaison life a lot easier. OK, I'm probably too selfish on that.
 ;) - it will make stable maintainers life a lot easier. The main
 reason why stable maintainers and distributions like recent oslo
 graduation movement is that we don't need to track each bug fix we
 need in every project, and waste lots of cycles on it. Being able
 to fix a bug in one place only is *highly* anticipated. [OK, I'm
 quite selfish on that one too.] - it's a delusion that there will
 be no neutron-main syncs that will break neutron-*aas repos ever.
 There can still be problems due to incompatibility between neutron
 main and neutron-*aas code resulted EXACTLY because multiple parts
 of the same process use different versions of the same module.
 
 That said, Doug Wiegley (lbaas core) seems to be in favour of
 option 3. due to lower coupling 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Doug Hellmann

On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

 Signed PGP part
 On 15/12/14 18:57, Doug Hellmann wrote:
  There may be a similar problem managing dependencies on libraries
  that live outside of either tree. I assume you already decided how
  to handle that. Are you doing the same thing, and adding the
  requirements to neutron’s lists?
 
 I guess the idea is to keep in neutron-*aas only those oslo-incubator
 modules that are used there solely (=not used in main repo).

How are the *aas packages installed? Are they separate libraries or 
applications that are installed on top of neutron? Or are their files copied 
into the neutron namespace?

 
 I think requirements are a bit easier and should track all direct
 dependencies in each of the repos, so that in case main repo decides
 to drop one, neutron-*aas repos are not broken.
 
 For requirements, it's different because there is no major burden due
 to duplicate entries in repos.
 
 
  On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com
  wrote:
 
  Hi all,
 
  Ihar and I discussed this on IRC, and are going forward with
  option 2 unless someone has a big problem with it.
 
  Thanks, Doug
 
 
  On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com
  wrote:
 
  Hi Ihar,
 
  I’m actually in favor of option 2, but it implies a few things
  about your time, and I wanted to chat with you before
  presuming.
 
  Maintenance can not involve breaking changes. At this point,
  the co-gate will block it.  Also, oslo graduation changes will
  have to be made in the services repos first, and then Neutron.
 
  Thanks, doug
 
 
  On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com
  wrote:
 
  Hi all,
 
  the question arose recently in one of reviews for neutron-*aas
  repos to remove all oslo-incubator code from those repos since
  it's duplicated in neutron main repo. (You can find the link to the
  review at the end of the email.)
 
  Brief hostory: neutron repo was recently split into 4 pieces
  (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split
  resulted in each repository keeping their own copy of
  neutron/openstack/common/... tree (currently unused in all
  neutron-*aas repos that are still bound to modules from main
  repo).
 
  As a oslo liaison for the project, I wonder what's the best way to
  manage oslo-incubator files. We have several options:
 
  1. just kill all the neutron/openstack/common/ trees from
  neutron-*aas repositories and continue using modules from main
  repo.
 
  2. kill all duplicate modules from neutron-*aas repos and leave
  only those that are used in those repos but not in main repo.
 
  3. fully duplicate all those modules in each of four repos that use
  them.
 
  I think option 1. is a straw man, since we should be able to
  introduce new oslo-incubator modules into neutron-*aas repos even
  if they are not used in main repo.
 
  Option 2. is good when it comes to synching non-breaking bug fixes
  (or security fixes) from oslo-incubator, in that it will require
  only one sync patch instead of e.g. four. At the same time there
  may be potential issues when synchronizing updates from
  oslo-incubator that would break API and hence require changes to
  each of the modules that use it. Since we don't support atomic
  merges for multiple projects in gate, we will need to be cautious
  about those updates, and we will still need to leave neutron-*aas
  repos broken for some time (though the time may be mitigated with
  care).
 
  Option 3. is vice versa - in theory, you get total decoupling,
  meaning no oslo-incubator updates in main repo are expected to
  break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
  I would vote for option 2., for two reasons: - most oslo-incubator
  syncs are non-breaking, and we may effectively apply care to
  updates that may result in potential breakage (f.e. being able to
  trigger an integrated run for each of neutron-*aas repos with the
  main sync patch, if there are any concerns). - it will make oslo
  liaison life a lot easier. OK, I'm probably too selfish on that.
  ;) - it will make stable maintainers life a lot easier. The main
  reason why stable maintainers and distributions like recent oslo
  graduation movement is that we don't need to track each bug fix we
  need in every project, and waste lots of cycles on it. Being able
  to fix a bug in one place only is *highly* anticipated. [OK, I'm
  quite selfish on that one too.] - it's a delusion that there will
  be no neutron-main syncs that will break neutron-*aas repos ever.
  There can still be problems due to incompatibility between neutron
  main and neutron-*aas code resulted EXACTLY because multiple parts
  of the same process use different versions of the same module.
 
  That said, Doug Wiegley (lbaas core) seems to be in favour of
  option 3. due to lower coupling that is achieved in that way. I
  know that lbaas team had a bad experience due to tight 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Doug Hellmann

On Dec 16, 2014, at 5:22 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

 Signed PGP part
 On 15/12/14 17:22, Doug Wiegley wrote:
  Hi Ihar,
 
  I’m actually in favor of option 2, but it implies a few things
  about your time, and I wanted to chat with you before presuming.
 
 I think split didn't mean moving project trees under separate
 governance, so I assume oslo (doc, qa, ...) liaisons should not be
 split either.
 
 
  Maintenance can not involve breaking changes. At this point, the
  co-gate will block it.  Also, oslo graduation changes will have to
  be made in the services repos first, and then Neutron.
 
 Do you mean that a change to oslo-incubator modules is co-gated (not
 just co-checked with no vote) with each of advanced services?
 
 As I pointed in my previous email, sometimes breakages are inescapable.
 
 Consider a change to neutron oslo-incubator module used commonly in
 all repos that breaks API (they are quite rare, but still have a
 chance of happening once in a while). If we would co-gate main neutron
 repo changes with services, it will mean that we won't be able to
 merge the change.
 
 That would probably suggest that we go forward with option 3 and
 manage all incubator files separately in each of the trees, though,
 again, breakages are still possible in that scenario via introducing
 incompatibility between versions of incubator modules in separate repos.
 
 So we should be realistic about it and plan forward how we deal
 potential breakages that *may* occur.
 
 As for oslo library graduations, the order is not really significant.
 What is significant is that we drop oslo-incubator module from main
 neutron repo only after all other neutron-*aas repos migrate to
 appropriate oslo.* library. The neutron migration itself may occur in
 parallel (by postponing module drop later).

Don’t assume that it’s safe to combine the incubated version and library 
version of a module. We’ve had some examples where the APIs change or global 
state changes in a way that make the two incompatible. We definitely don’t take 
any care to ensure that the two copies can be run together.

 
 
  Thanks, doug
 
 
  On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com
  wrote:
 
  Hi all,
 
  the question arose recently in one of reviews for neutron-*aas
  repos to remove all oslo-incubator code from those repos since
  it's duplicated in neutron main repo. (You can find the link to the
  review at the end of the email.)
 
  Brief hostory: neutron repo was recently split into 4 pieces
  (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split
  resulted in each repository keeping their own copy of
  neutron/openstack/common/... tree (currently unused in all
  neutron-*aas repos that are still bound to modules from main
  repo).
 
  As a oslo liaison for the project, I wonder what's the best way to
  manage oslo-incubator files. We have several options:
 
  1. just kill all the neutron/openstack/common/ trees from
  neutron-*aas repositories and continue using modules from main
  repo.
 
  2. kill all duplicate modules from neutron-*aas repos and leave
  only those that are used in those repos but not in main repo.
 
  3. fully duplicate all those modules in each of four repos that use
  them.
 
  I think option 1. is a straw man, since we should be able to
  introduce new oslo-incubator modules into neutron-*aas repos even
  if they are not used in main repo.
 
  Option 2. is good when it comes to synching non-breaking bug fixes
  (or security fixes) from oslo-incubator, in that it will require
  only one sync patch instead of e.g. four. At the same time there
  may be potential issues when synchronizing updates from
  oslo-incubator that would break API and hence require changes to
  each of the modules that use it. Since we don't support atomic
  merges for multiple projects in gate, we will need to be cautious
  about those updates, and we will still need to leave neutron-*aas
  repos broken for some time (though the time may be mitigated with
  care).
 
  Option 3. is vice versa - in theory, you get total decoupling,
  meaning no oslo-incubator updates in main repo are expected to
  break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
  I would vote for option 2., for two reasons: - most oslo-incubator
  syncs are non-breaking, and we may effectively apply care to
  updates that may result in potential breakage (f.e. being able to
  trigger an integrated run for each of neutron-*aas repos with the
  main sync patch, if there are any concerns). - it will make oslo
  liaison life a lot easier. OK, I'm probably too selfish on that.
  ;) - it will make stable maintainers life a lot easier. The main
  reason why stable maintainers and distributions like recent oslo
  graduation movement is that we don't need to track each bug fix we
  need in every project, and waste lots of cycles on it. Being able
  to fix a bug in one place only is *highly* anticipated. [OK, I'm
  quite 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 16/12/14 12:50, Doug Hellmann wrote:
 
 On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com
 wrote:
 
 Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote:
 There may be a similar problem managing dependencies on
 libraries that live outside of either tree. I assume you
 already decided how to handle that. Are you doing the same
 thing, and adding the requirements to neutron’s lists?
 
 I guess the idea is to keep in neutron-*aas only those
 oslo-incubator modules that are used there solely (=not used in
 main repo).
 
 How are the *aas packages installed? Are they separate libraries or
 applications that are installed on top of neutron? Or are their
 files copied into the neutron namespace?

They are separate libraries with their own setup.py, dependencies,
tarballs, all that, but they are free to use (public) code from main
neutron package.

 
 
 I think requirements are a bit easier and should track all
 direct dependencies in each of the repos, so that in case main
 repo decides to drop one, neutron-*aas repos are not broken.
 
 For requirements, it's different because there is no major burden
 due to duplicate entries in repos.
 
 
 On Dec 15, 2014, at 12:16 PM, Doug Wiegley
 do...@a10networks.com wrote:
 
 Hi all,
 
 Ihar and I discussed this on IRC, and are going forward with 
 option 2 unless someone has a big problem with it.
 
 Thanks, Doug
 
 
 On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com 
 wrote:
 
 Hi Ihar,
 
 I’m actually in favor of option 2, but it implies a few
 things about your time, and I wanted to chat with you
 before presuming.
 
 Maintenance can not involve breaking changes. At this
 point, the co-gate will block it.  Also, oslo graduation
 changes will have to be made in the services repos first,
 and then Neutron.
 
 Thanks, doug
 
 
 On 12/15/14, 6:15 AM, Ihar Hrachyshka
 ihrac...@redhat.com wrote:
 
 Hi all,
 
 the question arose recently in one of reviews for neutron-*aas 
 repos to remove all oslo-incubator code from those repos since 
 it's duplicated in neutron main repo. (You can find the link to
 the review at the end of the email.)
 
 Brief hostory: neutron repo was recently split into 4 pieces 
 (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The
 split resulted in each repository keeping their own copy of 
 neutron/openstack/common/... tree (currently unused in all 
 neutron-*aas repos that are still bound to modules from main 
 repo).
 
 As a oslo liaison for the project, I wonder what's the best way
 to manage oslo-incubator files. We have several options:
 
 1. just kill all the neutron/openstack/common/ trees from 
 neutron-*aas repositories and continue using modules from main 
 repo.
 
 2. kill all duplicate modules from neutron-*aas repos and
 leave only those that are used in those repos but not in main
 repo.
 
 3. fully duplicate all those modules in each of four repos that
 use them.
 
 I think option 1. is a straw man, since we should be able to 
 introduce new oslo-incubator modules into neutron-*aas repos
 even if they are not used in main repo.
 
 Option 2. is good when it comes to synching non-breaking bug
 fixes (or security fixes) from oslo-incubator, in that it will
 require only one sync patch instead of e.g. four. At the same
 time there may be potential issues when synchronizing updates
 from oslo-incubator that would break API and hence require
 changes to each of the modules that use it. Since we don't
 support atomic merges for multiple projects in gate, we will
 need to be cautious about those updates, and we will still need
 to leave neutron-*aas repos broken for some time (though the
 time may be mitigated with care).
 
 Option 3. is vice versa - in theory, you get total decoupling, 
 meaning no oslo-incubator updates in main repo are expected to 
 break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
 I would vote for option 2., for two reasons: - most
 oslo-incubator syncs are non-breaking, and we may effectively
 apply care to updates that may result in potential breakage
 (f.e. being able to trigger an integrated run for each of
 neutron-*aas repos with the main sync patch, if there are any
 concerns). - it will make oslo liaison life a lot easier. OK,
 I'm probably too selfish on that. ;) - it will make stable
 maintainers life a lot easier. The main reason why stable
 maintainers and distributions like recent oslo graduation
 movement is that we don't need to track each bug fix we need in
 every project, and waste lots of cycles on it. Being able to
 fix a bug in one place only is *highly* anticipated. [OK, I'm 
 quite selfish on that one too.] - it's a delusion that there
 will be no neutron-main syncs that will break neutron-*aas
 repos ever. There can still be problems due to incompatibility
 between neutron main and neutron-*aas code resulted EXACTLY
 because multiple parts of the same process use different
 versions of the same 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Doug Hellmann

On Dec 16, 2014, at 7:27 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

 Signed PGP part
 On 16/12/14 12:50, Doug Hellmann wrote:
 
  On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com
  wrote:
 
  Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote:
  There may be a similar problem managing dependencies on
  libraries that live outside of either tree. I assume you
  already decided how to handle that. Are you doing the same
  thing, and adding the requirements to neutron’s lists?
 
  I guess the idea is to keep in neutron-*aas only those
  oslo-incubator modules that are used there solely (=not used in
  main repo).
 
  How are the *aas packages installed? Are they separate libraries or
  applications that are installed on top of neutron? Or are their
  files copied into the neutron namespace?
 
 They are separate libraries with their own setup.py, dependencies,
 tarballs, all that, but they are free to use (public) code from main
 neutron package.

OK.

If they don’t have copies of all of the incubated modules they use, how are 
they tested? Is neutron a dependency?

 
 
 
  I think requirements are a bit easier and should track all
  direct dependencies in each of the repos, so that in case main
  repo decides to drop one, neutron-*aas repos are not broken.
 
  For requirements, it's different because there is no major burden
  due to duplicate entries in repos.
 
 
  On Dec 15, 2014, at 12:16 PM, Doug Wiegley
  do...@a10networks.com wrote:
 
  Hi all,
 
  Ihar and I discussed this on IRC, and are going forward with
  option 2 unless someone has a big problem with it.
 
  Thanks, Doug
 
 
  On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com
  wrote:
 
  Hi Ihar,
 
  I’m actually in favor of option 2, but it implies a few
  things about your time, and I wanted to chat with you
  before presuming.
 
  Maintenance can not involve breaking changes. At this
  point, the co-gate will block it.  Also, oslo graduation
  changes will have to be made in the services repos first,
  and then Neutron.
 
  Thanks, doug
 
 
  On 12/15/14, 6:15 AM, Ihar Hrachyshka
  ihrac...@redhat.com wrote:
 
  Hi all,
 
  the question arose recently in one of reviews for neutron-*aas
  repos to remove all oslo-incubator code from those repos since
  it's duplicated in neutron main repo. (You can find the link to
  the review at the end of the email.)
 
  Brief hostory: neutron repo was recently split into 4 pieces
  (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The
  split resulted in each repository keeping their own copy of
  neutron/openstack/common/... tree (currently unused in all
  neutron-*aas repos that are still bound to modules from main
  repo).
 
  As a oslo liaison for the project, I wonder what's the best way
  to manage oslo-incubator files. We have several options:
 
  1. just kill all the neutron/openstack/common/ trees from
  neutron-*aas repositories and continue using modules from main
  repo.
 
  2. kill all duplicate modules from neutron-*aas repos and
  leave only those that are used in those repos but not in main
  repo.
 
  3. fully duplicate all those modules in each of four repos that
  use them.
 
  I think option 1. is a straw man, since we should be able to
  introduce new oslo-incubator modules into neutron-*aas repos
  even if they are not used in main repo.
 
  Option 2. is good when it comes to synching non-breaking bug
  fixes (or security fixes) from oslo-incubator, in that it will
  require only one sync patch instead of e.g. four. At the same
  time there may be potential issues when synchronizing updates
  from oslo-incubator that would break API and hence require
  changes to each of the modules that use it. Since we don't
  support atomic merges for multiple projects in gate, we will
  need to be cautious about those updates, and we will still need
  to leave neutron-*aas repos broken for some time (though the
  time may be mitigated with care).
 
  Option 3. is vice versa - in theory, you get total decoupling,
  meaning no oslo-incubator updates in main repo are expected to
  break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
  I would vote for option 2., for two reasons: - most
  oslo-incubator syncs are non-breaking, and we may effectively
  apply care to updates that may result in potential breakage
  (f.e. being able to trigger an integrated run for each of
  neutron-*aas repos with the main sync patch, if there are any
  concerns). - it will make oslo liaison life a lot easier. OK,
  I'm probably too selfish on that. ;) - it will make stable
  maintainers life a lot easier. The main reason why stable
  maintainers and distributions like recent oslo graduation
  movement is that we don't need to track each bug fix we need in
  every project, and waste lots of cycles on it. Being able to
  fix a bug in one place only is *highly* anticipated. [OK, I'm
  quite selfish on that one too.] - it's a delusion that there
  will be no neutron-main 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 16/12/14 12:52, Doug Hellmann wrote:
 
 On Dec 16, 2014, at 5:22 AM, Ihar Hrachyshka ihrac...@redhat.com
 wrote:
 
 Signed PGP part On 15/12/14 17:22, Doug Wiegley wrote:
 Hi Ihar,
 
 I’m actually in favor of option 2, but it implies a few things 
 about your time, and I wanted to chat with you before
 presuming.
 
 I think split didn't mean moving project trees under separate 
 governance, so I assume oslo (doc, qa, ...) liaisons should not
 be split either.
 
 
 Maintenance can not involve breaking changes. At this point,
 the co-gate will block it.  Also, oslo graduation changes will
 have to be made in the services repos first, and then Neutron.
 
 Do you mean that a change to oslo-incubator modules is co-gated
 (not just co-checked with no vote) with each of advanced
 services?
 
 As I pointed in my previous email, sometimes breakages are
 inescapable.
 
 Consider a change to neutron oslo-incubator module used commonly
 in all repos that breaks API (they are quite rare, but still have
 a chance of happening once in a while). If we would co-gate main
 neutron repo changes with services, it will mean that we won't be
 able to merge the change.
 
 That would probably suggest that we go forward with option 3 and 
 manage all incubator files separately in each of the trees,
 though, again, breakages are still possible in that scenario via
 introducing incompatibility between versions of incubator modules
 in separate repos.
 
 So we should be realistic about it and plan forward how we deal 
 potential breakages that *may* occur.
 
 As for oslo library graduations, the order is not really
 significant. What is significant is that we drop oslo-incubator
 module from main neutron repo only after all other neutron-*aas
 repos migrate to appropriate oslo.* library. The neutron
 migration itself may occur in parallel (by postponing module drop
 later).
 
 Don’t assume that it’s safe to combine the incubated version and
 library version of a module. We’ve had some examples where the APIs
 change or global state changes in a way that make the two
 incompatible. We definitely don’t take any care to ensure that the
 two copies can be run together.

Hm. Does it leave us with option 3 only? In that case, should we care
about incompatibilities between different versions of incubator
modules running in the same process (one for core code, and another
one for a service)? That sounds more like we're not left with safe
options.

 
 
 
 Thanks, doug
 
 
 On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com 
 wrote:
 
 Hi all,
 
 the question arose recently in one of reviews for neutron-*aas 
 repos to remove all oslo-incubator code from those repos since 
 it's duplicated in neutron main repo. (You can find the link to
 the review at the end of the email.)
 
 Brief hostory: neutron repo was recently split into 4 pieces 
 (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The
 split resulted in each repository keeping their own copy of 
 neutron/openstack/common/... tree (currently unused in all 
 neutron-*aas repos that are still bound to modules from main 
 repo).
 
 As a oslo liaison for the project, I wonder what's the best way
 to manage oslo-incubator files. We have several options:
 
 1. just kill all the neutron/openstack/common/ trees from 
 neutron-*aas repositories and continue using modules from main 
 repo.
 
 2. kill all duplicate modules from neutron-*aas repos and
 leave only those that are used in those repos but not in main
 repo.
 
 3. fully duplicate all those modules in each of four repos that
 use them.
 
 I think option 1. is a straw man, since we should be able to 
 introduce new oslo-incubator modules into neutron-*aas repos
 even if they are not used in main repo.
 
 Option 2. is good when it comes to synching non-breaking bug
 fixes (or security fixes) from oslo-incubator, in that it will
 require only one sync patch instead of e.g. four. At the same
 time there may be potential issues when synchronizing updates
 from oslo-incubator that would break API and hence require
 changes to each of the modules that use it. Since we don't
 support atomic merges for multiple projects in gate, we will
 need to be cautious about those updates, and we will still need
 to leave neutron-*aas repos broken for some time (though the
 time may be mitigated with care).
 
 Option 3. is vice versa - in theory, you get total decoupling, 
 meaning no oslo-incubator updates in main repo are expected to 
 break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
 I would vote for option 2., for two reasons: - most
 oslo-incubator syncs are non-breaking, and we may effectively
 apply care to updates that may result in potential breakage
 (f.e. being able to trigger an integrated run for each of
 neutron-*aas repos with the main sync patch, if there are any
 concerns). - it will make oslo liaison life a lot easier. OK,
 I'm probably too selfish on that. ;) 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 16/12/14 13:41, Doug Hellmann wrote:
 
 On Dec 16, 2014, at 7:27 AM, Ihar Hrachyshka ihrac...@redhat.com
 wrote:
 
 Signed PGP part On 16/12/14 12:50, Doug Hellmann wrote:
 
 On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka
 ihrac...@redhat.com wrote:
 
 Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote:
 There may be a similar problem managing dependencies on 
 libraries that live outside of either tree. I assume you 
 already decided how to handle that. Are you doing the same 
 thing, and adding the requirements to neutron’s lists?
 
 I guess the idea is to keep in neutron-*aas only those 
 oslo-incubator modules that are used there solely (=not used
 in main repo).
 
 How are the *aas packages installed? Are they separate
 libraries or applications that are installed on top of neutron?
 Or are their files copied into the neutron namespace?
 
 They are separate libraries with their own setup.py,
 dependencies, tarballs, all that, but they are free to use
 (public) code from main neutron package.
 
 OK.
 
 If they don’t have copies of all of the incubated modules they use,
 how are they tested? Is neutron a dependency?

Yes, neutron is in their requirements.txt files.

 
 
 
 
 I think requirements are a bit easier and should track all 
 direct dependencies in each of the repos, so that in case
 main repo decides to drop one, neutron-*aas repos are not
 broken.
 
 For requirements, it's different because there is no major
 burden due to duplicate entries in repos.
 
 
 On Dec 15, 2014, at 12:16 PM, Doug Wiegley 
 do...@a10networks.com wrote:
 
 Hi all,
 
 Ihar and I discussed this on IRC, and are going forward
 with option 2 unless someone has a big problem with it.
 
 Thanks, Doug
 
 
 On 12/15/14, 8:22 AM, Doug Wiegley
 do...@a10networks.com wrote:
 
 Hi Ihar,
 
 I’m actually in favor of option 2, but it implies a
 few things about your time, and I wanted to chat with
 you before presuming.
 
 Maintenance can not involve breaking changes. At this 
 point, the co-gate will block it.  Also, oslo
 graduation changes will have to be made in the services
 repos first, and then Neutron.
 
 Thanks, doug
 
 
 On 12/15/14, 6:15 AM, Ihar Hrachyshka 
 ihrac...@redhat.com wrote:
 
 Hi all,
 
 the question arose recently in one of reviews for
 neutron-*aas repos to remove all oslo-incubator code from
 those repos since it's duplicated in neutron main repo.
 (You can find the link to the review at the end of the
 email.)
 
 Brief hostory: neutron repo was recently split into 4
 pieces (main, neutron-fwaas, neutron-lbaas, and
 neutron-vpnaas). The split resulted in each repository
 keeping their own copy of neutron/openstack/common/... tree
 (currently unused in all neutron-*aas repos that are still
 bound to modules from main repo).
 
 As a oslo liaison for the project, I wonder what's the best
 way to manage oslo-incubator files. We have several
 options:
 
 1. just kill all the neutron/openstack/common/ trees from 
 neutron-*aas repositories and continue using modules from
 main repo.
 
 2. kill all duplicate modules from neutron-*aas repos and 
 leave only those that are used in those repos but not in
 main repo.
 
 3. fully duplicate all those modules in each of four repos
 that use them.
 
 I think option 1. is a straw man, since we should be able
 to introduce new oslo-incubator modules into neutron-*aas
 repos even if they are not used in main repo.
 
 Option 2. is good when it comes to synching non-breaking
 bug fixes (or security fixes) from oslo-incubator, in that
 it will require only one sync patch instead of e.g. four.
 At the same time there may be potential issues when
 synchronizing updates from oslo-incubator that would break
 API and hence require changes to each of the modules that
 use it. Since we don't support atomic merges for multiple
 projects in gate, we will need to be cautious about those
 updates, and we will still need to leave neutron-*aas repos
 broken for some time (though the time may be mitigated with
 care).
 
 Option 3. is vice versa - in theory, you get total
 decoupling, meaning no oslo-incubator updates in main repo
 are expected to break neutron-*aas repos, but bug fixing
 becomes a huge PITA.
 
 I would vote for option 2., for two reasons: - most 
 oslo-incubator syncs are non-breaking, and we may
 effectively apply care to updates that may result in
 potential breakage (f.e. being able to trigger an
 integrated run for each of neutron-*aas repos with the main
 sync patch, if there are any concerns). - it will make oslo
 liaison life a lot easier. OK, I'm probably too selfish on
 that. ;) - it will make stable maintainers life a lot
 easier. The main reason why stable maintainers and
 distributions like recent oslo graduation movement is that
 we don't need to track each bug fix we need in every
 project, and waste lots of cycles on it. Being able to fix
 a bug in one place only is *highly* anticipated. [OK, I'm 
 quite 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Gary Kotton


On 12/16/14, 2:41 PM, Doug Hellmann d...@doughellmann.com wrote:


On Dec 16, 2014, at 7:27 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

 Signed PGP part
 On 16/12/14 12:50, Doug Hellmann wrote:
 
  On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com
  wrote:
 
  Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote:
  There may be a similar problem managing dependencies on
  libraries that live outside of either tree. I assume you
  already decided how to handle that. Are you doing the same
  thing, and adding the requirements to neutron¹s lists?
 
  I guess the idea is to keep in neutron-*aas only those
  oslo-incubator modules that are used there solely (=not used in
  main repo).
 
  How are the *aas packages installed? Are they separate libraries or
  applications that are installed on top of neutron? Or are their
  files copied into the neutron namespace?
 
 They are separate libraries with their own setup.py, dependencies,
 tarballs, all that, but they are free to use (public) code from main
 neutron package.

OK.

If they don¹t have copies of all of the incubated modules they use, how
are they tested? Is neutron a dependency?

This is/was one of my concerns with the decomposition proposal. It is not
clear if neutron is a dependency. My two cents is that it should be.


 
 
 
  I think requirements are a bit easier and should track all
  direct dependencies in each of the repos, so that in case main
  repo decides to drop one, neutron-*aas repos are not broken.
 
  For requirements, it's different because there is no major burden
  due to duplicate entries in repos.
 
 
  On Dec 15, 2014, at 12:16 PM, Doug Wiegley
  do...@a10networks.com wrote:
 
  Hi all,
 
  Ihar and I discussed this on IRC, and are going forward with
  option 2 unless someone has a big problem with it.
 
  Thanks, Doug
 
 
  On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com
  wrote:
 
  Hi Ihar,
 
  I¹m actually in favor of option 2, but it implies a few
  things about your time, and I wanted to chat with you
  before presuming.
 
  Maintenance can not involve breaking changes. At this
  point, the co-gate will block it.  Also, oslo graduation
  changes will have to be made in the services repos first,
  and then Neutron.
 
  Thanks, doug
 
 
  On 12/15/14, 6:15 AM, Ihar Hrachyshka
  ihrac...@redhat.com wrote:
 
  Hi all,
 
  the question arose recently in one of reviews for neutron-*aas
  repos to remove all oslo-incubator code from those repos since
  it's duplicated in neutron main repo. (You can find the link to
  the review at the end of the email.)
 
  Brief hostory: neutron repo was recently split into 4 pieces
  (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The
  split resulted in each repository keeping their own copy of
  neutron/openstack/common/... tree (currently unused in all
  neutron-*aas repos that are still bound to modules from main
  repo).
 
  As a oslo liaison for the project, I wonder what's the best way
  to manage oslo-incubator files. We have several options:
 
  1. just kill all the neutron/openstack/common/ trees from
  neutron-*aas repositories and continue using modules from main
  repo.
 
  2. kill all duplicate modules from neutron-*aas repos and
  leave only those that are used in those repos but not in main
  repo.
 
  3. fully duplicate all those modules in each of four repos that
  use them.
 
  I think option 1. is a straw man, since we should be able to
  introduce new oslo-incubator modules into neutron-*aas repos
  even if they are not used in main repo.
 
  Option 2. is good when it comes to synching non-breaking bug
  fixes (or security fixes) from oslo-incubator, in that it will
  require only one sync patch instead of e.g. four. At the same
  time there may be potential issues when synchronizing updates
  from oslo-incubator that would break API and hence require
  changes to each of the modules that use it. Since we don't
  support atomic merges for multiple projects in gate, we will
  need to be cautious about those updates, and we will still need
  to leave neutron-*aas repos broken for some time (though the
  time may be mitigated with care).
 
  Option 3. is vice versa - in theory, you get total decoupling,
  meaning no oslo-incubator updates in main repo are expected to
  break neutron-*aas repos, but bug fixing becomes a huge PITA.
 
  I would vote for option 2., for two reasons: - most
  oslo-incubator syncs are non-breaking, and we may effectively
  apply care to updates that may result in potential breakage
  (f.e. being able to trigger an integrated run for each of
  neutron-*aas repos with the main sync patch, if there are any
  concerns). - it will make oslo liaison life a lot easier. OK,
  I'm probably too selfish on that. ;) - it will make stable
  maintainers life a lot easier. The main reason why stable
  maintainers and distributions like recent oslo graduation
  movement is that we don't need to track each bug fix we need in
  

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-16 Thread Doug Hellmann

On Dec 16, 2014, at 7:42 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

 Signed PGP part
 On 16/12/14 12:52, Doug Hellmann wrote:
 
  On Dec 16, 2014, at 5:22 AM, Ihar Hrachyshka ihrac...@redhat.com
  wrote:
 
  Signed PGP part On 15/12/14 17:22, Doug Wiegley wrote:
  Hi Ihar,
 
  I’m actually in favor of option 2, but it implies a few things
  about your time, and I wanted to chat with you before
  presuming.
 
  I think split didn't mean moving project trees under separate
  governance, so I assume oslo (doc, qa, ...) liaisons should not
  be split either.
 
 
  Maintenance can not involve breaking changes. At this point,
  the co-gate will block it.  Also, oslo graduation changes will
  have to be made in the services repos first, and then Neutron.
 
  Do you mean that a change to oslo-incubator modules is co-gated
  (not just co-checked with no vote) with each of advanced
  services?
 
  As I pointed in my previous email, sometimes breakages are
  inescapable.
 
  Consider a change to neutron oslo-incubator module used commonly
  in all repos that breaks API (they are quite rare, but still have
  a chance of happening once in a while). If we would co-gate main
  neutron repo changes with services, it will mean that we won't be
  able to merge the change.
 
  That would probably suggest that we go forward with option 3 and
  manage all incubator files separately in each of the trees,
  though, again, breakages are still possible in that scenario via
  introducing incompatibility between versions of incubator modules
  in separate repos.
 
  So we should be realistic about it and plan forward how we deal
  potential breakages that *may* occur.
 
  As for oslo library graduations, the order is not really
  significant. What is significant is that we drop oslo-incubator
  module from main neutron repo only after all other neutron-*aas
  repos migrate to appropriate oslo.* library. The neutron
  migration itself may occur in parallel (by postponing module drop
  later).
 
  Don’t assume that it’s safe to combine the incubated version and
  library version of a module. We’ve had some examples where the APIs
  change or global state changes in a way that make the two
  incompatible. We definitely don’t take any care to ensure that the
  two copies can be run together.
 
 Hm. Does it leave us with option 3 only? In that case, should we care
 about incompatibilities between different versions of incubator
 modules running in the same process (one for core code, and another
 one for a service)? That sounds more like we're not left with safe
 options.

I think you only want to have one copy of the Oslo modules active in a process 
at any given point. That probably means having the *aas projects use whatever 
incubated Oslo modules are in the main neutron repository instead of their own 
copy, but as you point out that will break those projects when neutron adopts a 
new library. You might end up having to build shims in neutron to hide the Oslo 
change during the transition.

OTOH, it may not be a big deal. We don’t go out of our way to break 
compatibility, so you might find that it works fine in a lot of cases. I think 
context won’t, because it holds global state, but some of the others should be 
fine.

FWIW, usually when we hit a dependency problem like this, the solution is to 
split one of the projects up so there is a library that can be used by all of 
the consumers. It sounds like neutron is trying to be both an application and a 
library.

 
 
 
 
  Thanks, doug
 
 
  On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com
  wrote:
 
  Hi all,
 
  the question arose recently in one of reviews for neutron-*aas
  repos to remove all oslo-incubator code from those repos since
  it's duplicated in neutron main repo. (You can find the link to
  the review at the end of the email.)
 
  Brief hostory: neutron repo was recently split into 4 pieces
  (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The
  split resulted in each repository keeping their own copy of
  neutron/openstack/common/... tree (currently unused in all
  neutron-*aas repos that are still bound to modules from main
  repo).
 
  As a oslo liaison for the project, I wonder what's the best way
  to manage oslo-incubator files. We have several options:
 
  1. just kill all the neutron/openstack/common/ trees from
  neutron-*aas repositories and continue using modules from main
  repo.
 
  2. kill all duplicate modules from neutron-*aas repos and
  leave only those that are used in those repos but not in main
  repo.
 
  3. fully duplicate all those modules in each of four repos that
  use them.
 
  I think option 1. is a straw man, since we should be able to
  introduce new oslo-incubator modules into neutron-*aas repos
  even if they are not used in main repo.
 
  Option 2. is good when it comes to synching non-breaking bug
  fixes (or security fixes) from oslo-incubator, in that it will
  require only one sync patch instead of 

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-15 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 15/12/14 15:15, Ihar Hrachyshka wrote:
 - it's a delusion that there will be no neutron-main syncs that
 will break neutron-*aas repos ever.

OK, I've just decided to check whether my (non-native speaker)
understanding of the meaning of the word 'delusion' is correct, and I
need to admit that what I've found out in dictionaries is not what I
really meant. :|

I only meant that it's a wrong assumption.

So in case anyone reads it as any kind of derogation, I'm very sorry
for the bad wording. Please blame my bad English. And Richard Dawkins.

/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUju7sAAoJEC5aWaUY1u57XfIH/ituoB51dLLaLvryFumADvT/
dhkJyzylD/x0j++BS88KdNE9i7aiAFn2MQyvINxYV7THSsgl60YruV6xXj5X72aK
EUd967OI77XuIheOP6iIC2ZoGa3ie8RGyMTxbTW5hEeDR8+mtYhQTmDZUWKtT15o
jviGV9/kPftVU2U1UirwjpZY3DPee4D9CwIoJdTKvk93+NNNlMh1cAsWIR0ISJBC
mm/X020SSl2wOy9d3lUge4QEi698NPYpkAAbAqL6YTkblXOFgfb7EBexGQoV388P
TFb2StHyCD7hVpdx6ljLWR2mEQVavIJE9VUkvAzvoBMmlZnYFFx4TnUEu6Vu7VY=
=Q+GY
-END PGP SIGNATURE-

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-15 Thread Doug Wiegley
Hi Ihar,

I’m actually in favor of option 2, but it implies a few things about your
time, and I wanted to chat with you before presuming.

Maintenance can not involve breaking changes. At this point, the co-gate
will block it.  Also, oslo graduation changes will have to be made in the
services repos first, and then Neutron.

Thanks,
doug


On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi all,

the question arose recently in one of reviews for neutron-*aas repos
to remove all oslo-incubator code from those repos since it's
duplicated in neutron main repo. (You can find the link to the review
at the end of the email.)

Brief hostory: neutron repo was recently split into 4 pieces (main,
neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted
in each repository keeping their own copy of
neutron/openstack/common/... tree (currently unused in all
neutron-*aas repos that are still bound to modules from main repo).

As a oslo liaison for the project, I wonder what's the best way to
manage oslo-incubator files. We have several options:

1. just kill all the neutron/openstack/common/ trees from neutron-*aas
repositories and continue using modules from main repo.

2. kill all duplicate modules from neutron-*aas repos and leave only
those that are used in those repos but not in main repo.

3. fully duplicate all those modules in each of four repos that use them.

I think option 1. is a straw man, since we should be able to introduce
new oslo-incubator modules into neutron-*aas repos even if they are
not used in main repo.

Option 2. is good when it comes to synching non-breaking bug fixes (or
security fixes) from oslo-incubator, in that it will require only one
sync patch instead of e.g. four. At the same time there may be
potential issues when synchronizing updates from oslo-incubator that
would break API and hence require changes to each of the modules that
use it. Since we don't support atomic merges for multiple projects in
gate, we will need to be cautious about those updates, and we will
still need to leave neutron-*aas repos broken for some time (though
the time may be mitigated with care).

Option 3. is vice versa - in theory, you get total decoupling, meaning
no oslo-incubator updates in main repo are expected to break
neutron-*aas repos, but bug fixing becomes a huge PITA.

I would vote for option 2., for two reasons:
- - most oslo-incubator syncs are non-breaking, and we may effectively
apply care to updates that may result in potential breakage (f.e.
being able to trigger an integrated run for each of neutron-*aas repos
with the main sync patch, if there are any concerns).
- - it will make oslo liaison life a lot easier. OK, I'm probably too
selfish on that. ;)
- - it will make stable maintainers life a lot easier. The main reason
why stable maintainers and distributions like recent oslo graduation
movement is that we don't need to track each bug fix we need in every
project, and waste lots of cycles on it. Being able to fix a bug in
one place only is *highly* anticipated. [OK, I'm quite selfish on that
one too.]
- - it's a delusion that there will be no neutron-main syncs that will
break neutron-*aas repos ever. There can still be problems due to
incompatibility between neutron main and neutron-*aas code resulted
EXACTLY because multiple parts of the same process use different
versions of the same module.

That said, Doug Wiegley (lbaas core) seems to be in favour of option
3. due to lower coupling that is achieved in that way. I know that
lbaas team had a bad experience due to tight coupling to neutron
project in the past, so I appreciate their concerns.

All in all, we should come up with some standard solution for both
advanced services that are already split out, *and* upcoming vendor
plugin shrinking initiative.

The initial discussion is captured at:
https://review.openstack.org/#/c/141427/

Thanks,
/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh
apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq
6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6
tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E
QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/
czLUCyr/nJg4aw8Qm0DTjnZxS+BBe5De0Ke4zm2AGePgFYcai8YQPtuOfSJDbXk=
=D6Gn
-END PGP SIGNATURE-

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-15 Thread Doug Wiegley
Hi all,

Ihar and I discussed this on IRC, and are going forward with option 2
unless someone has a big problem with it.

Thanks,
Doug


On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote:

Hi Ihar,

I’m actually in favor of option 2, but it implies a few things about your
time, and I wanted to chat with you before presuming.

Maintenance can not involve breaking changes. At this point, the co-gate
will block it.  Also, oslo graduation changes will have to be made in the
services repos first, and then Neutron.

Thanks,
doug


On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi all,

the question arose recently in one of reviews for neutron-*aas repos
to remove all oslo-incubator code from those repos since it's
duplicated in neutron main repo. (You can find the link to the review
at the end of the email.)

Brief hostory: neutron repo was recently split into 4 pieces (main,
neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted
in each repository keeping their own copy of
neutron/openstack/common/... tree (currently unused in all
neutron-*aas repos that are still bound to modules from main repo).

As a oslo liaison for the project, I wonder what's the best way to
manage oslo-incubator files. We have several options:

1. just kill all the neutron/openstack/common/ trees from neutron-*aas
repositories and continue using modules from main repo.

2. kill all duplicate modules from neutron-*aas repos and leave only
those that are used in those repos but not in main repo.

3. fully duplicate all those modules in each of four repos that use them.

I think option 1. is a straw man, since we should be able to introduce
new oslo-incubator modules into neutron-*aas repos even if they are
not used in main repo.

Option 2. is good when it comes to synching non-breaking bug fixes (or
security fixes) from oslo-incubator, in that it will require only one
sync patch instead of e.g. four. At the same time there may be
potential issues when synchronizing updates from oslo-incubator that
would break API and hence require changes to each of the modules that
use it. Since we don't support atomic merges for multiple projects in
gate, we will need to be cautious about those updates, and we will
still need to leave neutron-*aas repos broken for some time (though
the time may be mitigated with care).

Option 3. is vice versa - in theory, you get total decoupling, meaning
no oslo-incubator updates in main repo are expected to break
neutron-*aas repos, but bug fixing becomes a huge PITA.

I would vote for option 2., for two reasons:
- - most oslo-incubator syncs are non-breaking, and we may effectively
apply care to updates that may result in potential breakage (f.e.
being able to trigger an integrated run for each of neutron-*aas repos
with the main sync patch, if there are any concerns).
- - it will make oslo liaison life a lot easier. OK, I'm probably too
selfish on that. ;)
- - it will make stable maintainers life a lot easier. The main reason
why stable maintainers and distributions like recent oslo graduation
movement is that we don't need to track each bug fix we need in every
project, and waste lots of cycles on it. Being able to fix a bug in
one place only is *highly* anticipated. [OK, I'm quite selfish on that
one too.]
- - it's a delusion that there will be no neutron-main syncs that will
break neutron-*aas repos ever. There can still be problems due to
incompatibility between neutron main and neutron-*aas code resulted
EXACTLY because multiple parts of the same process use different
versions of the same module.

That said, Doug Wiegley (lbaas core) seems to be in favour of option
3. due to lower coupling that is achieved in that way. I know that
lbaas team had a bad experience due to tight coupling to neutron
project in the past, so I appreciate their concerns.

All in all, we should come up with some standard solution for both
advanced services that are already split out, *and* upcoming vendor
plugin shrinking initiative.

The initial discussion is captured at:
https://review.openstack.org/#/c/141427/

Thanks,
/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh
apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq
6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6
tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E
QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/
czLUCyr/nJg4aw8Qm0DTjnZxS+BBe5De0Ke4zm2AGePgFYcai8YQPtuOfSJDbXk=
=D6Gn
-END PGP SIGNATURE-


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-15 Thread Kyle Mestery
Option 2 works for me, thanks for figuring this out Ihar and Doug!

On Mon, Dec 15, 2014 at 11:16 AM, Doug Wiegley do...@a10networks.com
wrote:

 Hi all,

 Ihar and I discussed this on IRC, and are going forward with option 2
 unless someone has a big problem with it.

 Thanks,
 Doug


 On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote:

 Hi Ihar,
 
 I’m actually in favor of option 2, but it implies a few things about your
 time, and I wanted to chat with you before presuming.
 
 Maintenance can not involve breaking changes. At this point, the co-gate
 will block it.  Also, oslo graduation changes will have to be made in the
 services repos first, and then Neutron.
 
 Thanks,
 doug
 
 
 On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:
 
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512
 
 Hi all,
 
 the question arose recently in one of reviews for neutron-*aas repos
 to remove all oslo-incubator code from those repos since it's
 duplicated in neutron main repo. (You can find the link to the review
 at the end of the email.)
 
 Brief hostory: neutron repo was recently split into 4 pieces (main,
 neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted
 in each repository keeping their own copy of
 neutron/openstack/common/... tree (currently unused in all
 neutron-*aas repos that are still bound to modules from main repo).
 
 As a oslo liaison for the project, I wonder what's the best way to
 manage oslo-incubator files. We have several options:
 
 1. just kill all the neutron/openstack/common/ trees from neutron-*aas
 repositories and continue using modules from main repo.
 
 2. kill all duplicate modules from neutron-*aas repos and leave only
 those that are used in those repos but not in main repo.
 
 3. fully duplicate all those modules in each of four repos that use them.
 
 I think option 1. is a straw man, since we should be able to introduce
 new oslo-incubator modules into neutron-*aas repos even if they are
 not used in main repo.
 
 Option 2. is good when it comes to synching non-breaking bug fixes (or
 security fixes) from oslo-incubator, in that it will require only one
 sync patch instead of e.g. four. At the same time there may be
 potential issues when synchronizing updates from oslo-incubator that
 would break API and hence require changes to each of the modules that
 use it. Since we don't support atomic merges for multiple projects in
 gate, we will need to be cautious about those updates, and we will
 still need to leave neutron-*aas repos broken for some time (though
 the time may be mitigated with care).
 
 Option 3. is vice versa - in theory, you get total decoupling, meaning
 no oslo-incubator updates in main repo are expected to break
 neutron-*aas repos, but bug fixing becomes a huge PITA.
 
 I would vote for option 2., for two reasons:
 - - most oslo-incubator syncs are non-breaking, and we may effectively
 apply care to updates that may result in potential breakage (f.e.
 being able to trigger an integrated run for each of neutron-*aas repos
 with the main sync patch, if there are any concerns).
 - - it will make oslo liaison life a lot easier. OK, I'm probably too
 selfish on that. ;)
 - - it will make stable maintainers life a lot easier. The main reason
 why stable maintainers and distributions like recent oslo graduation
 movement is that we don't need to track each bug fix we need in every
 project, and waste lots of cycles on it. Being able to fix a bug in
 one place only is *highly* anticipated. [OK, I'm quite selfish on that
 one too.]
 - - it's a delusion that there will be no neutron-main syncs that will
 break neutron-*aas repos ever. There can still be problems due to
 incompatibility between neutron main and neutron-*aas code resulted
 EXACTLY because multiple parts of the same process use different
 versions of the same module.
 
 That said, Doug Wiegley (lbaas core) seems to be in favour of option
 3. due to lower coupling that is achieved in that way. I know that
 lbaas team had a bad experience due to tight coupling to neutron
 project in the past, so I appreciate their concerns.
 
 All in all, we should come up with some standard solution for both
 advanced services that are already split out, *and* upcoming vendor
 plugin shrinking initiative.
 
 The initial discussion is captured at:
 https://review.openstack.org/#/c/141427/
 
 Thanks,
 /Ihar
 -BEGIN PGP SIGNATURE-
 Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
 
 iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh
 apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq
 6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6
 tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E
 QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/
 czLUCyr/nJg4aw8Qm0DTjnZxS+BBe5De0Ke4zm2AGePgFYcai8YQPtuOfSJDbXk=
 =D6Gn
 -END PGP SIGNATURE-
 

 ___
 OpenStack-dev mailing list

Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split

2014-12-15 Thread Doug Hellmann
There may be a similar problem managing dependencies on libraries that live 
outside of either tree. I assume you already decided how to handle that. Are 
you doing the same thing, and adding the requirements to neutron’s lists?

On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote:

 Hi all,
 
 Ihar and I discussed this on IRC, and are going forward with option 2
 unless someone has a big problem with it.
 
 Thanks,
 Doug
 
 
 On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote:
 
 Hi Ihar,
 
 I’m actually in favor of option 2, but it implies a few things about your
 time, and I wanted to chat with you before presuming.
 
 Maintenance can not involve breaking changes. At this point, the co-gate
 will block it.  Also, oslo graduation changes will have to be made in the
 services repos first, and then Neutron.
 
 Thanks,
 doug
 
 
 On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:
 
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512
 
 Hi all,
 
 the question arose recently in one of reviews for neutron-*aas repos
 to remove all oslo-incubator code from those repos since it's
 duplicated in neutron main repo. (You can find the link to the review
 at the end of the email.)
 
 Brief hostory: neutron repo was recently split into 4 pieces (main,
 neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted
 in each repository keeping their own copy of
 neutron/openstack/common/... tree (currently unused in all
 neutron-*aas repos that are still bound to modules from main repo).
 
 As a oslo liaison for the project, I wonder what's the best way to
 manage oslo-incubator files. We have several options:
 
 1. just kill all the neutron/openstack/common/ trees from neutron-*aas
 repositories and continue using modules from main repo.
 
 2. kill all duplicate modules from neutron-*aas repos and leave only
 those that are used in those repos but not in main repo.
 
 3. fully duplicate all those modules in each of four repos that use them.
 
 I think option 1. is a straw man, since we should be able to introduce
 new oslo-incubator modules into neutron-*aas repos even if they are
 not used in main repo.
 
 Option 2. is good when it comes to synching non-breaking bug fixes (or
 security fixes) from oslo-incubator, in that it will require only one
 sync patch instead of e.g. four. At the same time there may be
 potential issues when synchronizing updates from oslo-incubator that
 would break API and hence require changes to each of the modules that
 use it. Since we don't support atomic merges for multiple projects in
 gate, we will need to be cautious about those updates, and we will
 still need to leave neutron-*aas repos broken for some time (though
 the time may be mitigated with care).
 
 Option 3. is vice versa - in theory, you get total decoupling, meaning
 no oslo-incubator updates in main repo are expected to break
 neutron-*aas repos, but bug fixing becomes a huge PITA.
 
 I would vote for option 2., for two reasons:
 - - most oslo-incubator syncs are non-breaking, and we may effectively
 apply care to updates that may result in potential breakage (f.e.
 being able to trigger an integrated run for each of neutron-*aas repos
 with the main sync patch, if there are any concerns).
 - - it will make oslo liaison life a lot easier. OK, I'm probably too
 selfish on that. ;)
 - - it will make stable maintainers life a lot easier. The main reason
 why stable maintainers and distributions like recent oslo graduation
 movement is that we don't need to track each bug fix we need in every
 project, and waste lots of cycles on it. Being able to fix a bug in
 one place only is *highly* anticipated. [OK, I'm quite selfish on that
 one too.]
 - - it's a delusion that there will be no neutron-main syncs that will
 break neutron-*aas repos ever. There can still be problems due to
 incompatibility between neutron main and neutron-*aas code resulted
 EXACTLY because multiple parts of the same process use different
 versions of the same module.
 
 That said, Doug Wiegley (lbaas core) seems to be in favour of option
 3. due to lower coupling that is achieved in that way. I know that
 lbaas team had a bad experience due to tight coupling to neutron
 project in the past, so I appreciate their concerns.
 
 All in all, we should come up with some standard solution for both
 advanced services that are already split out, *and* upcoming vendor
 plugin shrinking initiative.
 
 The initial discussion is captured at:
 https://review.openstack.org/#/c/141427/
 
 Thanks,
 /Ihar
 -BEGIN PGP SIGNATURE-
 Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
 
 iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh
 apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq
 6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6
 tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E
 QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/