Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 15/12/14 18:57, Doug Hellmann wrote: There may be a similar problem managing dependencies on libraries that live outside of either tree. I assume you already decided how to handle that. Are you doing the same thing, and adding the requirements to neutron’s lists? I guess the idea is to keep in neutron-*aas only those oslo-incubator modules that are used there solely (=not used in main repo). I think requirements are a bit easier and should track all direct dependencies in each of the repos, so that in case main repo decides to drop one, neutron-*aas repos are not broken. For requirements, it's different because there is no major burden due to duplicate entries in repos. On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling that is achieved in that way. I know that lbaas team had a bad experience due to tight coupling to neutron project in the past, so I appreciate their concerns. All in all, we should come up with some standard solution for both advanced services that are already split out, *and* upcoming vendor plugin shrinking initiative. The initial discussion is captured at:
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 15/12/14 17:22, Doug Wiegley wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. I think split didn't mean moving project trees under separate governance, so I assume oslo (doc, qa, ...) liaisons should not be split either. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Do you mean that a change to oslo-incubator modules is co-gated (not just co-checked with no vote) with each of advanced services? As I pointed in my previous email, sometimes breakages are inescapable. Consider a change to neutron oslo-incubator module used commonly in all repos that breaks API (they are quite rare, but still have a chance of happening once in a while). If we would co-gate main neutron repo changes with services, it will mean that we won't be able to merge the change. That would probably suggest that we go forward with option 3 and manage all incubator files separately in each of the trees, though, again, breakages are still possible in that scenario via introducing incompatibility between versions of incubator modules in separate repos. So we should be realistic about it and plan forward how we deal potential breakages that *may* occur. As for oslo library graduations, the order is not really significant. What is significant is that we drop oslo-incubator module from main neutron repo only after all other neutron-*aas repos migrate to appropriate oslo.* library. The neutron migration itself may occur in parallel (by postponing module drop later). Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote: There may be a similar problem managing dependencies on libraries that live outside of either tree. I assume you already decided how to handle that. Are you doing the same thing, and adding the requirements to neutron’s lists? I guess the idea is to keep in neutron-*aas only those oslo-incubator modules that are used there solely (=not used in main repo). How are the *aas packages installed? Are they separate libraries or applications that are installed on top of neutron? Or are their files copied into the neutron namespace? I think requirements are a bit easier and should track all direct dependencies in each of the repos, so that in case main repo decides to drop one, neutron-*aas repos are not broken. For requirements, it's different because there is no major burden due to duplicate entries in repos. On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling that is achieved in that way. I know that lbaas team had a bad experience due to tight
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
On Dec 16, 2014, at 5:22 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 17:22, Doug Wiegley wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. I think split didn't mean moving project trees under separate governance, so I assume oslo (doc, qa, ...) liaisons should not be split either. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Do you mean that a change to oslo-incubator modules is co-gated (not just co-checked with no vote) with each of advanced services? As I pointed in my previous email, sometimes breakages are inescapable. Consider a change to neutron oslo-incubator module used commonly in all repos that breaks API (they are quite rare, but still have a chance of happening once in a while). If we would co-gate main neutron repo changes with services, it will mean that we won't be able to merge the change. That would probably suggest that we go forward with option 3 and manage all incubator files separately in each of the trees, though, again, breakages are still possible in that scenario via introducing incompatibility between versions of incubator modules in separate repos. So we should be realistic about it and plan forward how we deal potential breakages that *may* occur. As for oslo library graduations, the order is not really significant. What is significant is that we drop oslo-incubator module from main neutron repo only after all other neutron-*aas repos migrate to appropriate oslo.* library. The neutron migration itself may occur in parallel (by postponing module drop later). Don’t assume that it’s safe to combine the incubated version and library version of a module. We’ve had some examples where the APIs change or global state changes in a way that make the two incompatible. We definitely don’t take any care to ensure that the two copies can be run together. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 16/12/14 12:50, Doug Hellmann wrote: On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote: There may be a similar problem managing dependencies on libraries that live outside of either tree. I assume you already decided how to handle that. Are you doing the same thing, and adding the requirements to neutron’s lists? I guess the idea is to keep in neutron-*aas only those oslo-incubator modules that are used there solely (=not used in main repo). How are the *aas packages installed? Are they separate libraries or applications that are installed on top of neutron? Or are their files copied into the neutron namespace? They are separate libraries with their own setup.py, dependencies, tarballs, all that, but they are free to use (public) code from main neutron package. I think requirements are a bit easier and should track all direct dependencies in each of the repos, so that in case main repo decides to drop one, neutron-*aas repos are not broken. For requirements, it's different because there is no major burden due to duplicate entries in repos. On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
On Dec 16, 2014, at 7:27 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 16/12/14 12:50, Doug Hellmann wrote: On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote: There may be a similar problem managing dependencies on libraries that live outside of either tree. I assume you already decided how to handle that. Are you doing the same thing, and adding the requirements to neutron’s lists? I guess the idea is to keep in neutron-*aas only those oslo-incubator modules that are used there solely (=not used in main repo). How are the *aas packages installed? Are they separate libraries or applications that are installed on top of neutron? Or are their files copied into the neutron namespace? They are separate libraries with their own setup.py, dependencies, tarballs, all that, but they are free to use (public) code from main neutron package. OK. If they don’t have copies of all of the incubated modules they use, how are they tested? Is neutron a dependency? I think requirements are a bit easier and should track all direct dependencies in each of the repos, so that in case main repo decides to drop one, neutron-*aas repos are not broken. For requirements, it's different because there is no major burden due to duplicate entries in repos. On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - it's a delusion that there will be no neutron-main
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 16/12/14 12:52, Doug Hellmann wrote: On Dec 16, 2014, at 5:22 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 17:22, Doug Wiegley wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. I think split didn't mean moving project trees under separate governance, so I assume oslo (doc, qa, ...) liaisons should not be split either. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Do you mean that a change to oslo-incubator modules is co-gated (not just co-checked with no vote) with each of advanced services? As I pointed in my previous email, sometimes breakages are inescapable. Consider a change to neutron oslo-incubator module used commonly in all repos that breaks API (they are quite rare, but still have a chance of happening once in a while). If we would co-gate main neutron repo changes with services, it will mean that we won't be able to merge the change. That would probably suggest that we go forward with option 3 and manage all incubator files separately in each of the trees, though, again, breakages are still possible in that scenario via introducing incompatibility between versions of incubator modules in separate repos. So we should be realistic about it and plan forward how we deal potential breakages that *may* occur. As for oslo library graduations, the order is not really significant. What is significant is that we drop oslo-incubator module from main neutron repo only after all other neutron-*aas repos migrate to appropriate oslo.* library. The neutron migration itself may occur in parallel (by postponing module drop later). Don’t assume that it’s safe to combine the incubated version and library version of a module. We’ve had some examples where the APIs change or global state changes in a way that make the two incompatible. We definitely don’t take any care to ensure that the two copies can be run together. Hm. Does it leave us with option 3 only? In that case, should we care about incompatibilities between different versions of incubator modules running in the same process (one for core code, and another one for a service)? That sounds more like we're not left with safe options. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;)
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 16/12/14 13:41, Doug Hellmann wrote: On Dec 16, 2014, at 7:27 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 16/12/14 12:50, Doug Hellmann wrote: On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote: There may be a similar problem managing dependencies on libraries that live outside of either tree. I assume you already decided how to handle that. Are you doing the same thing, and adding the requirements to neutron’s lists? I guess the idea is to keep in neutron-*aas only those oslo-incubator modules that are used there solely (=not used in main repo). How are the *aas packages installed? Are they separate libraries or applications that are installed on top of neutron? Or are their files copied into the neutron namespace? They are separate libraries with their own setup.py, dependencies, tarballs, all that, but they are free to use (public) code from main neutron package. OK. If they don’t have copies of all of the incubated modules they use, how are they tested? Is neutron a dependency? Yes, neutron is in their requirements.txt files. I think requirements are a bit easier and should track all direct dependencies in each of the repos, so that in case main repo decides to drop one, neutron-*aas repos are not broken. For requirements, it's different because there is no major burden due to duplicate entries in repos. On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
On 12/16/14, 2:41 PM, Doug Hellmann d...@doughellmann.com wrote: On Dec 16, 2014, at 7:27 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 16/12/14 12:50, Doug Hellmann wrote: On Dec 16, 2014, at 5:13 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 18:57, Doug Hellmann wrote: There may be a similar problem managing dependencies on libraries that live outside of either tree. I assume you already decided how to handle that. Are you doing the same thing, and adding the requirements to neutron¹s lists? I guess the idea is to keep in neutron-*aas only those oslo-incubator modules that are used there solely (=not used in main repo). How are the *aas packages installed? Are they separate libraries or applications that are installed on top of neutron? Or are their files copied into the neutron namespace? They are separate libraries with their own setup.py, dependencies, tarballs, all that, but they are free to use (public) code from main neutron package. OK. If they don¹t have copies of all of the incubated modules they use, how are they tested? Is neutron a dependency? This is/was one of my concerns with the decomposition proposal. It is not clear if neutron is a dependency. My two cents is that it should be. I think requirements are a bit easier and should track all direct dependencies in each of the repos, so that in case main repo decides to drop one, neutron-*aas repos are not broken. For requirements, it's different because there is no major burden due to duplicate entries in repos. On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I¹m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
On Dec 16, 2014, at 7:42 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 16/12/14 12:52, Doug Hellmann wrote: On Dec 16, 2014, at 5:22 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 15/12/14 17:22, Doug Wiegley wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. I think split didn't mean moving project trees under separate governance, so I assume oslo (doc, qa, ...) liaisons should not be split either. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Do you mean that a change to oslo-incubator modules is co-gated (not just co-checked with no vote) with each of advanced services? As I pointed in my previous email, sometimes breakages are inescapable. Consider a change to neutron oslo-incubator module used commonly in all repos that breaks API (they are quite rare, but still have a chance of happening once in a while). If we would co-gate main neutron repo changes with services, it will mean that we won't be able to merge the change. That would probably suggest that we go forward with option 3 and manage all incubator files separately in each of the trees, though, again, breakages are still possible in that scenario via introducing incompatibility between versions of incubator modules in separate repos. So we should be realistic about it and plan forward how we deal potential breakages that *may* occur. As for oslo library graduations, the order is not really significant. What is significant is that we drop oslo-incubator module from main neutron repo only after all other neutron-*aas repos migrate to appropriate oslo.* library. The neutron migration itself may occur in parallel (by postponing module drop later). Don’t assume that it’s safe to combine the incubated version and library version of a module. We’ve had some examples where the APIs change or global state changes in a way that make the two incompatible. We definitely don’t take any care to ensure that the two copies can be run together. Hm. Does it leave us with option 3 only? In that case, should we care about incompatibilities between different versions of incubator modules running in the same process (one for core code, and another one for a service)? That sounds more like we're not left with safe options. I think you only want to have one copy of the Oslo modules active in a process at any given point. That probably means having the *aas projects use whatever incubated Oslo modules are in the main neutron repository instead of their own copy, but as you point out that will break those projects when neutron adopts a new library. You might end up having to build shims in neutron to hide the Oslo change during the transition. OTOH, it may not be a big deal. We don’t go out of our way to break compatibility, so you might find that it works fine in a lot of cases. I think context won’t, because it holds global state, but some of the others should be fine. FWIW, usually when we hit a dependency problem like this, the solution is to split one of the projects up so there is a library that can be used by all of the consumers. It sounds like neutron is trying to be both an application and a library. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of
[openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling that is achieved in that way. I know that lbaas team had a bad experience due to tight coupling to neutron project in the past, so I appreciate their concerns. All in all, we should come up with some standard solution for both advanced services that are already split out, *and* upcoming vendor plugin shrinking initiative. The initial discussion is captured at: https://review.openstack.org/#/c/141427/ Thanks, /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq 6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6 tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/ czLUCyr/nJg4aw8Qm0DTjnZxS+BBe5De0Ke4zm2AGePgFYcai8YQPtuOfSJDbXk= =D6Gn -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 15/12/14 15:15, Ihar Hrachyshka wrote: - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. OK, I've just decided to check whether my (non-native speaker) understanding of the meaning of the word 'delusion' is correct, and I need to admit that what I've found out in dictionaries is not what I really meant. :| I only meant that it's a wrong assumption. So in case anyone reads it as any kind of derogation, I'm very sorry for the bad wording. Please blame my bad English. And Richard Dawkins. /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUju7sAAoJEC5aWaUY1u57XfIH/ituoB51dLLaLvryFumADvT/ dhkJyzylD/x0j++BS88KdNE9i7aiAFn2MQyvINxYV7THSsgl60YruV6xXj5X72aK EUd967OI77XuIheOP6iIC2ZoGa3ie8RGyMTxbTW5hEeDR8+mtYhQTmDZUWKtT15o jviGV9/kPftVU2U1UirwjpZY3DPee4D9CwIoJdTKvk93+NNNlMh1cAsWIR0ISJBC mm/X020SSl2wOy9d3lUge4QEi698NPYpkAAbAqL6YTkblXOFgfb7EBexGQoV388P TFb2StHyCD7hVpdx6ljLWR2mEQVavIJE9VUkvAzvoBMmlZnYFFx4TnUEu6Vu7VY= =Q+GY -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling that is achieved in that way. I know that lbaas team had a bad experience due to tight coupling to neutron project in the past, so I appreciate their concerns. All in all, we should come up with some standard solution for both advanced services that are already split out, *and* upcoming vendor plugin shrinking initiative. The initial discussion is captured at: https://review.openstack.org/#/c/141427/ Thanks, /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq 6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6 tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/ czLUCyr/nJg4aw8Qm0DTjnZxS+BBe5De0Ke4zm2AGePgFYcai8YQPtuOfSJDbXk= =D6Gn -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling that is achieved in that way. I know that lbaas team had a bad experience due to tight coupling to neutron project in the past, so I appreciate their concerns. All in all, we should come up with some standard solution for both advanced services that are already split out, *and* upcoming vendor plugin shrinking initiative. The initial discussion is captured at: https://review.openstack.org/#/c/141427/ Thanks, /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq 6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6 tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/ czLUCyr/nJg4aw8Qm0DTjnZxS+BBe5De0Ke4zm2AGePgFYcai8YQPtuOfSJDbXk= =D6Gn -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
Option 2 works for me, thanks for figuring this out Ihar and Doug! On Mon, Dec 15, 2014 at 11:16 AM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling that is achieved in that way. I know that lbaas team had a bad experience due to tight coupling to neutron project in the past, so I appreciate their concerns. All in all, we should come up with some standard solution for both advanced services that are already split out, *and* upcoming vendor plugin shrinking initiative. The initial discussion is captured at: https://review.openstack.org/#/c/141427/ Thanks, /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq 6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6 tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/ czLUCyr/nJg4aw8Qm0DTjnZxS+BBe5De0Ke4zm2AGePgFYcai8YQPtuOfSJDbXk= =D6Gn -END PGP SIGNATURE- ___ OpenStack-dev mailing list
Re: [openstack-dev] [all][oslo][neutron] Managing oslo-incubator modules after project split
There may be a similar problem managing dependencies on libraries that live outside of either tree. I assume you already decided how to handle that. Are you doing the same thing, and adding the requirements to neutron’s lists? On Dec 15, 2014, at 12:16 PM, Doug Wiegley do...@a10networks.com wrote: Hi all, Ihar and I discussed this on IRC, and are going forward with option 2 unless someone has a big problem with it. Thanks, Doug On 12/15/14, 8:22 AM, Doug Wiegley do...@a10networks.com wrote: Hi Ihar, I’m actually in favor of option 2, but it implies a few things about your time, and I wanted to chat with you before presuming. Maintenance can not involve breaking changes. At this point, the co-gate will block it. Also, oslo graduation changes will have to be made in the services repos first, and then Neutron. Thanks, doug On 12/15/14, 6:15 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, the question arose recently in one of reviews for neutron-*aas repos to remove all oslo-incubator code from those repos since it's duplicated in neutron main repo. (You can find the link to the review at the end of the email.) Brief hostory: neutron repo was recently split into 4 pieces (main, neutron-fwaas, neutron-lbaas, and neutron-vpnaas). The split resulted in each repository keeping their own copy of neutron/openstack/common/... tree (currently unused in all neutron-*aas repos that are still bound to modules from main repo). As a oslo liaison for the project, I wonder what's the best way to manage oslo-incubator files. We have several options: 1. just kill all the neutron/openstack/common/ trees from neutron-*aas repositories and continue using modules from main repo. 2. kill all duplicate modules from neutron-*aas repos and leave only those that are used in those repos but not in main repo. 3. fully duplicate all those modules in each of four repos that use them. I think option 1. is a straw man, since we should be able to introduce new oslo-incubator modules into neutron-*aas repos even if they are not used in main repo. Option 2. is good when it comes to synching non-breaking bug fixes (or security fixes) from oslo-incubator, in that it will require only one sync patch instead of e.g. four. At the same time there may be potential issues when synchronizing updates from oslo-incubator that would break API and hence require changes to each of the modules that use it. Since we don't support atomic merges for multiple projects in gate, we will need to be cautious about those updates, and we will still need to leave neutron-*aas repos broken for some time (though the time may be mitigated with care). Option 3. is vice versa - in theory, you get total decoupling, meaning no oslo-incubator updates in main repo are expected to break neutron-*aas repos, but bug fixing becomes a huge PITA. I would vote for option 2., for two reasons: - - most oslo-incubator syncs are non-breaking, and we may effectively apply care to updates that may result in potential breakage (f.e. being able to trigger an integrated run for each of neutron-*aas repos with the main sync patch, if there are any concerns). - - it will make oslo liaison life a lot easier. OK, I'm probably too selfish on that. ;) - - it will make stable maintainers life a lot easier. The main reason why stable maintainers and distributions like recent oslo graduation movement is that we don't need to track each bug fix we need in every project, and waste lots of cycles on it. Being able to fix a bug in one place only is *highly* anticipated. [OK, I'm quite selfish on that one too.] - - it's a delusion that there will be no neutron-main syncs that will break neutron-*aas repos ever. There can still be problems due to incompatibility between neutron main and neutron-*aas code resulted EXACTLY because multiple parts of the same process use different versions of the same module. That said, Doug Wiegley (lbaas core) seems to be in favour of option 3. due to lower coupling that is achieved in that way. I know that lbaas team had a bad experience due to tight coupling to neutron project in the past, so I appreciate their concerns. All in all, we should come up with some standard solution for both advanced services that are already split out, *and* upcoming vendor plugin shrinking initiative. The initial discussion is captured at: https://review.openstack.org/#/c/141427/ Thanks, /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUju0NAAoJEC5aWaUY1u57n5YH/jA4l5DsLgRpw9gYsoSWVGvh apmJ4UlnAKhxzc787XImz1VA+ztSyIwAUdEdcfq3gkinP58q7o48oIXOGjFXaBNq 6qBePC1hflEqZ85Hm4/i5z51qutjW0dyi4y4C6FHgM5NsEkhbh0QIa/u8Hr4F1q6 tkr0kDbCbDkiZ8IX1l74VGWQ3QvCNeJkANUg79BqGq+qIVP3BeOHyWqRmpLZFQ6E QiUwhiYv5l4HekCEQN8PWisJoqnhbTNjvLBnLD82IitLd5vXnsXfSkxKhv9XeOg/
