Re: [openstack-dev] [oslo] change to deprecation policy in the incubator
On Aug 29, 2014, at 6:28 PM, Ben Nemec openst...@nemebean.com wrote: On 08/28/2014 11:14 AM, Doug Hellmann wrote: Before Juno we set a deprecation policy for graduating libraries that said the incubated versions of the modules would stay in the incubator repository for one full cycle after graduation. This gives projects time to adopt the libraries and still receive bug fixes to the incubated version (see https://wiki.openstack.org/wiki/Oslo#Graduation). That policy worked well early on, but has recently introduced some challenges with the low level modules. Other modules in the incubator are still importing the incubated versions of, for example, timeutils, and so tests that rely on mocking out or modifying the behavior of timeutils do not work as expected when different parts of the application code end up calling different versions of timeutils. We had similar issues with the notifiers and RPC code, and I expect to find other cases as we continue with the graduations. To deal with this problem, I propose that for Kilo we delete graduating modules as soon as the new library is released, rather than waiting to the end of the cycle. We can update the other incubated modules at the same time, so that the incubator will always use the new libraries and be consistent. So from a consumer perspective, this means projects will need to sync from stable/juno until they adopt the new libs and then they need to sync from master, which will also be using the new libs. That’s right. I would expect the sync to be part of the adoption process. One thing I think is worth noting is the fact that this will require projects to adopt all of the libs at once (or at least all of the libs that need to match incubator, but that's not always obvious so probably safest to just say all). It might be possible to sync some modules from master and some from stable, but that sounds like a mess waiting to happen. :-) Quite. I guess my concern here is that I don't think projects have been adopting all of the oslo libs at once, so if, for example, a project was looking at adopting oslo.i18n and oslo.utils they may have to do both at the same time since adopting one will require them to start syncing from master, and then they won't have the ability to use the graduated modules anymore. This may be a necessary evil, but it does raise the short-term bar for adopting any oslo lib, even if the end result will be the same (all of the released libs adopted). True, more below. We have not had a lot of patches where backports were necessary, but there have been a few important ones, so we need to retain the ability to handle them and allow projects to adopt libraries at a reasonable pace. To handle backports cleanly, we can “freeze” all changes to the master branch version of modules slated for graduation during Kilo (we would need to make a good list very early in the cycle), and use the stable/juno branch for backports. The new process would be: 1. Declare which modules we expect to graduate during Kilo. 2. Changes to those pre-graduation modules could be made in the master branch before their library is released, as long as the change is also backported to the stable/juno branch at the same time (we should enforce this by having both patches submitted before accepting either). 3. When graduation for a library starts, freeze those modules in all branches until the library is released. 4. Remove modules from the incubator’s master branch after the library is released. 5. Land changes in the library first. 6. Backport changes, as needed, to stable/juno instead of master. It would be better to begin the export/import process as early as possible in Kilo to keep the window where point 2 applies very short. If there are objections to using stable/juno, we could introduce a new branch with a name like backports/kilo, but I am afraid having the extra branch to manage would just cause confusion. I would like to move ahead with this plan by creating the stable/juno branch and starting to update the incubator as soon as the oslo.log repository is imported (https://review.openstack.org/116934). Thoughts? I think the obvious concern for me is the extra overhead of trying to keep one more branch in sync with all the others. With this we will require two commits for each change to incubator code that isn't graduating. Backporting to Havana would require four changes. I guess this is no worse than the situation with graduating code (one commit to the lib and one to incubator), but that's temporary pain for specific files. This would continue indefinitely for all files in incubator. We could probably help this by requiring changes to be linked in their commit messages so reviewers can vote on both changes at once, but it's still additional work for everyone so I think it's worth bringing up. I don't have a
Re: [openstack-dev] [oslo] change to deprecation policy in the incubator
On Aug 29, 2014, at 5:53 AM, Flavio Percoco fla...@redhat.com wrote: On 08/28/2014 06:14 PM, Doug Hellmann wrote: Before Juno we set a deprecation policy for graduating libraries that said the incubated versions of the modules would stay in the incubator repository for one full cycle after graduation. This gives projects time to adopt the libraries and still receive bug fixes to the incubated version (see https://wiki.openstack.org/wiki/Oslo#Graduation). That policy worked well early on, but has recently introduced some challenges with the low level modules. Other modules in the incubator are still importing the incubated versions of, for example, timeutils, and so tests that rely on mocking out or modifying the behavior of timeutils do not work as expected when different parts of the application code end up calling different versions of timeutils. We had similar issues with the notifiers and RPC code, and I expect to find other cases as we continue with the graduations. To deal with this problem, I propose that for Kilo we delete graduating modules as soon as the new library is released, rather than waiting to the end of the cycle. We can update the other incubated modules at the same time, so that the incubator will always use the new libraries and be consistent. We have not had a lot of patches where backports were necessary, but there have been a few important ones, so we need to retain the ability to handle them and allow projects to adopt libraries at a reasonable pace. To handle backports cleanly, we can “freeze” all changes to the master branch version of modules slated for graduation during Kilo (we would need to make a good list very early in the cycle), and use the stable/juno branch for backports. The new process would be: 1. Declare which modules we expect to graduate during Kilo. 2. Changes to those pre-graduation modules could be made in the master branch before their library is released, as long as the change is also backported to the stable/juno branch at the same time (we should enforce this by having both patches submitted before accepting either). 3. When graduation for a library starts, freeze those modules in all branches until the library is released. 4. Remove modules from the incubator’s master branch after the library is released. 5. Land changes in the library first. 6. Backport changes, as needed, to stable/juno instead of master. It would be better to begin the export/import process as early as possible in Kilo to keep the window where point 2 applies very short. If there are objections to using stable/juno, we could introduce a new branch with a name like backports/kilo, but I am afraid having the extra branch to manage would just cause confusion. I would like to move ahead with this plan by creating the stable/juno branch and starting to update the incubator as soon as the oslo.log repository is imported (https://review.openstack.org/116934). Thoughts? I like the plan. By being more aggressive in the way we deprecate graduated modules from oslo-incubator helps making sure the projects are all aligned. One thing we may want to think about is to graduate fewer modules in order to give liaisons enough time to migrate the projects they're taking care of. The more libs we graduate, the more work we're putting on liaisons, which means they'll need more time (besides the time they're dedicating to other projects) to do that work. I think the libs we’ll be working on in Kilo are a bit more complicated than what we’ve done in Juno, so that’s likely to happen as a natural consequence. We also don’t expect projects to adopt the libraries automatically in the cycle where they are graduated (that was the original intent behind delaying when we delete the code from the repository). One more thing, we need to add to the list of ports to do during Kilo the backlog of ports that haven't happened yet. For example, I haven't ported glance to oslo.utils yet. I expect to do it before the end of the cycle but Murphy :) You raise a good point, that we need to audit which projects are using code that has now graduated, and work with the liaisons from those projects on patches and reviews. It would be good if we could get caught up by K1 or K2 at the latest. I’m already working on checking the libraries we have released to ensure we finished all of the steps after that initial release. Does anyone else want to volunteer to do the audit of consuming projects? Doug Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] change to deprecation policy in the incubator
That all makes sense to me. Doug Hellmann wrote: Before Juno we set a deprecation policy for graduating libraries that said the incubated versions of the modules would stay in the incubator repository for one full cycle after graduation. This gives projects time to adopt the libraries and still receive bug fixes to the incubated version (see https://wiki.openstack.org/wiki/Oslo#Graduation). That policy worked well early on, but has recently introduced some challenges with the low level modules. Other modules in the incubator are still importing the incubated versions of, for example, timeutils, and so tests that rely on mocking out or modifying the behavior of timeutils do not work as expected when different parts of the application code end up calling different versions of timeutils. We had similar issues with the notifiers and RPC code, and I expect to find other cases as we continue with the graduations. To deal with this problem, I propose that for Kilo we delete graduating modules as soon as the new library is released, rather than waiting to the end of the cycle. We can update the other incubated modules at the same time, so that the incubator will always use the new libraries and be consistent. We have not had a lot of patches where backports were necessary, but there have been a few important ones, so we need to retain the ability to handle them and allow projects to adopt libraries at a reasonable pace. To handle backports cleanly, we can “freeze” all changes to the master branch version of modules slated for graduation during Kilo (we would need to make a good list very early in the cycle), and use the stable/juno branch for backports. The new process would be: 1. Declare which modules we expect to graduate during Kilo. 2. Changes to those pre-graduation modules could be made in the master branch before their library is released, as long as the change is also backported to the stable/juno branch at the same time (we should enforce this by having both patches submitted before accepting either). 3. When graduation for a library starts, freeze those modules in all branches until the library is released. 4. Remove modules from the incubator’s master branch after the library is released. 5. Land changes in the library first. 6. Backport changes, as needed, to stable/juno instead of master. It would be better to begin the export/import process as early as possible in Kilo to keep the window where point 2 applies very short. If there are objections to using stable/juno, we could introduce a new branch with a name like backports/kilo, but I am afraid having the extra branch to manage would just cause confusion. I would like to move ahead with this plan by creating the stable/juno branch and starting to update the incubator as soon as the oslo.log repository is imported (https://review.openstack.org/116934). Thoughts? Doug ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] change to deprecation policy in the incubator
On 08/28/2014 06:14 PM, Doug Hellmann wrote: Before Juno we set a deprecation policy for graduating libraries that said the incubated versions of the modules would stay in the incubator repository for one full cycle after graduation. This gives projects time to adopt the libraries and still receive bug fixes to the incubated version (see https://wiki.openstack.org/wiki/Oslo#Graduation). That policy worked well early on, but has recently introduced some challenges with the low level modules. Other modules in the incubator are still importing the incubated versions of, for example, timeutils, and so tests that rely on mocking out or modifying the behavior of timeutils do not work as expected when different parts of the application code end up calling different versions of timeutils. We had similar issues with the notifiers and RPC code, and I expect to find other cases as we continue with the graduations. To deal with this problem, I propose that for Kilo we delete graduating modules as soon as the new library is released, rather than waiting to the end of the cycle. We can update the other incubated modules at the same time, so that the incubator will always use the new libraries and be consistent. We have not had a lot of patches where backports were necessary, but there have been a few important ones, so we need to retain the ability to handle them and allow projects to adopt libraries at a reasonable pace. To handle backports cleanly, we can “freeze” all changes to the master branch version of modules slated for graduation during Kilo (we would need to make a good list very early in the cycle), and use the stable/juno branch for backports. The new process would be: 1. Declare which modules we expect to graduate during Kilo. 2. Changes to those pre-graduation modules could be made in the master branch before their library is released, as long as the change is also backported to the stable/juno branch at the same time (we should enforce this by having both patches submitted before accepting either). 3. When graduation for a library starts, freeze those modules in all branches until the library is released. 4. Remove modules from the incubator’s master branch after the library is released. 5. Land changes in the library first. 6. Backport changes, as needed, to stable/juno instead of master. It would be better to begin the export/import process as early as possible in Kilo to keep the window where point 2 applies very short. If there are objections to using stable/juno, we could introduce a new branch with a name like backports/kilo, but I am afraid having the extra branch to manage would just cause confusion. I would like to move ahead with this plan by creating the stable/juno branch and starting to update the incubator as soon as the oslo.log repository is imported (https://review.openstack.org/116934). Thoughts? I like the plan. By being more aggressive in the way we deprecate graduated modules from oslo-incubator helps making sure the projects are all aligned. One thing we may want to think about is to graduate fewer modules in order to give liaisons enough time to migrate the projects they're taking care of. The more libs we graduate, the more work we're putting on liaisons, which means they'll need more time (besides the time they're dedicating to other projects) to do that work. One more thing, we need to add to the list of ports to do during Kilo the backlog of ports that haven't happened yet. For example, I haven't ported glance to oslo.utils yet. I expect to do it before the end of the cycle but Murphy :) Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] change to deprecation policy in the incubator
On 08/28/2014 11:14 AM, Doug Hellmann wrote: Before Juno we set a deprecation policy for graduating libraries that said the incubated versions of the modules would stay in the incubator repository for one full cycle after graduation. This gives projects time to adopt the libraries and still receive bug fixes to the incubated version (see https://wiki.openstack.org/wiki/Oslo#Graduation). That policy worked well early on, but has recently introduced some challenges with the low level modules. Other modules in the incubator are still importing the incubated versions of, for example, timeutils, and so tests that rely on mocking out or modifying the behavior of timeutils do not work as expected when different parts of the application code end up calling different versions of timeutils. We had similar issues with the notifiers and RPC code, and I expect to find other cases as we continue with the graduations. To deal with this problem, I propose that for Kilo we delete graduating modules as soon as the new library is released, rather than waiting to the end of the cycle. We can update the other incubated modules at the same time, so that the incubator will always use the new libraries and be consistent. So from a consumer perspective, this means projects will need to sync from stable/juno until they adopt the new libs and then they need to sync from master, which will also be using the new libs. One thing I think is worth noting is the fact that this will require projects to adopt all of the libs at once (or at least all of the libs that need to match incubator, but that's not always obvious so probably safest to just say all). It might be possible to sync some modules from master and some from stable, but that sounds like a mess waiting to happen. :-) I guess my concern here is that I don't think projects have been adopting all of the oslo libs at once, so if, for example, a project was looking at adopting oslo.i18n and oslo.utils they may have to do both at the same time since adopting one will require them to start syncing from master, and then they won't have the ability to use the graduated modules anymore. This may be a necessary evil, but it does raise the short-term bar for adopting any oslo lib, even if the end result will be the same (all of the released libs adopted). We have not had a lot of patches where backports were necessary, but there have been a few important ones, so we need to retain the ability to handle them and allow projects to adopt libraries at a reasonable pace. To handle backports cleanly, we can “freeze” all changes to the master branch version of modules slated for graduation during Kilo (we would need to make a good list very early in the cycle), and use the stable/juno branch for backports. The new process would be: 1. Declare which modules we expect to graduate during Kilo. 2. Changes to those pre-graduation modules could be made in the master branch before their library is released, as long as the change is also backported to the stable/juno branch at the same time (we should enforce this by having both patches submitted before accepting either). 3. When graduation for a library starts, freeze those modules in all branches until the library is released. 4. Remove modules from the incubator’s master branch after the library is released. 5. Land changes in the library first. 6. Backport changes, as needed, to stable/juno instead of master. It would be better to begin the export/import process as early as possible in Kilo to keep the window where point 2 applies very short. If there are objections to using stable/juno, we could introduce a new branch with a name like backports/kilo, but I am afraid having the extra branch to manage would just cause confusion. I would like to move ahead with this plan by creating the stable/juno branch and starting to update the incubator as soon as the oslo.log repository is imported (https://review.openstack.org/116934). Thoughts? I think the obvious concern for me is the extra overhead of trying to keep one more branch in sync with all the others. With this we will require two commits for each change to incubator code that isn't graduating. Backporting to Havana would require four changes. I guess this is no worse than the situation with graduating code (one commit to the lib and one to incubator), but that's temporary pain for specific files. This would continue indefinitely for all files in incubator. We could probably help this by requiring changes to be linked in their commit messages so reviewers can vote on both changes at once, but it's still additional work for everyone so I think it's worth bringing up. I don't have a better solution to the incubator-lib mismatch issue so I'm okay with going forward on this, but it will introduce some new issues that I think we should be aware of going in. Doug
Re: [openstack-dev] [oslo] change to deprecation policy in the incubator
On Aug 28, 2014, at 12:14 PM, Doug Hellmann d...@doughellmann.com wrote: Before Juno we set a deprecation policy for graduating libraries that said the incubated versions of the modules would stay in the incubator repository for one full cycle after graduation. This gives projects time to adopt the libraries and still receive bug fixes to the incubated version (see https://wiki.openstack.org/wiki/Oslo#Graduation). That policy worked well early on, but has recently introduced some challenges with the low level modules. Other modules in the incubator are still importing the incubated versions of, for example, timeutils, and so tests that rely on mocking out or modifying the behavior of timeutils do not work as expected when different parts of the application code end up calling different versions of timeutils. We had similar issues with the notifiers and RPC code, and I expect to find other cases as we continue with the graduations. To deal with this problem, I propose that for Kilo we delete graduating modules as soon as the new library is released, rather than waiting to the end of the cycle. We can update the other incubated modules at the same time, so that the incubator will always use the new libraries and be consistent. We have not had a lot of patches where backports were necessary, but there have been a few important ones, so we need to retain the ability to handle them and allow projects to adopt libraries at a reasonable pace. To handle backports cleanly, we can “freeze” all changes to the master branch version of modules slated for graduation during Kilo (we would need to make a good list very early in the cycle), and use the stable/juno branch for backports. The new process would be: 1. Declare which modules we expect to graduate during Kilo. 2. Changes to those pre-graduation modules could be made in the master branch before their library is released, as long as the change is also backported to the stable/juno branch at the same time (we should enforce this by having both patches submitted before accepting either). 3. When graduation for a library starts, freeze those modules in all branches until the library is released. 4. Remove modules from the incubator’s master branch after the library is released. 5. Land changes in the library first. 6. Backport changes, as needed, to stable/juno instead of master. It would be better to begin the export/import process as early as possible in Kilo to keep the window where point 2 applies very short. If there are objections to using stable/juno, we could introduce a new branch with a name like backports/kilo, but I am afraid having the extra branch to manage would just cause confusion. I would like to move ahead with this plan by creating the stable/juno branch and starting to update the incubator as soon as the oslo.log repository is imported (https://review.openstack.org/116934). That change has merged and the oslo.log repository has been created. Doug Thoughts? Doug ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
