Re: release blockers for pip
Donald is handling the 9.0.x series. Paul just mentioned in IRC that he'll be doing the 10.x beta in 2 weeks time. Pip 9.0.2 is out, and the only change it carries is that it supports TLSv1.2 when running under system Python on macOS < 10.13. Official release notes: https://pip.pypa.io/en/stable/news/ Context: * https://pyfound.blogspot.com/2017/01/time-to-upgrade-your-python-tls-v12.html * https://github.com/pypa/warehouse/issues/3293 * https://status.python.org/incidents/btjtz01lzp88 -- Sumana Harihareswara Changeset Consulting https://changeset.nyc On 03/07/2018 03:13 PM, Paul Moore wrote: > Cool, that's good to know. The biggest admin issue I saw was tracking > "what's suitable for a maintenance release", but certainly my > impression is coloured by the big changes that went on since 9.0.1. > > Paul > > On 7 March 2018 at 17:59, Donald Stufft wrote: >> >> On Mar 7, 2018, at 12:39 PM, Paul Moore wrote: >> >> At the moment,we don't have the infrastructure for doing bugfix >> releases - and in this specific situation, pulling out the "ready to >> go" parts of master to form an interim release isn't really practical, >> given the resources we have. Once pip 10 is out of the door, I'd like >> to investigate the possibility of having some sort of "maintenance >> branch" setup, but we're so thin on the ground at the moment (with >> Donald working on Warehouse and Xavier on leave of absence, it's >> basically just Pradyun and I, and I'm not managing to actually work on >> code much, just reviews and issue management) so I don't want to >> overload what little resource we have with admin. >> >> >> >> Doing a maintenance release is only a little bit harder than doing a regular >> release and I don’t think that maintenance branches fix it. >> >> If we wanted to we could create a maintenance branch *right now* by just >> doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch >> off of whatever was released as 9.0.1 that we can cherry-pick changes to. I >> don’t think pre-creating this branch at release time adds anything of value >> (and in fact I think it makes the situation generally worse). >> >> * If changes land to master first and then get cherry-picked into a >> maintenance branch, then it’s basically no different from what is available >> today. >> * If changes land to the maintenance branch first, and then get forward >> merged to `master`, then people will get confused and send backwards >> incompatible changes to the maintenance branch and need to be asked to >> rebase their branch onto master. >> * Having the branch exist at all will confuse people who don’t know where to >> send what branch where. >> * In the past, we’ve had bugs get fixed in a maintenance branch, then forget >> to merge that into master and “lose” the bug fix. >> >> Basically, I think sending changes to the maintenance branch first makes >> contributing to pip more confusing and more likely we lose things by >> accident and sending things to `master` branch then asking for a cherry-pick >> to a maintenance branch isn’t really much less effort than collecting issues >> at a hypothetical “we want to release 9.0.2” time, creating a branch then, >> and cherry-picking them all over at that time. >> >> In either case, a 9.0.2 release is hard because we vastly altered the >> structure of the code between 9.0.1 and `master`, so either solution doesn’t >> really help us get a hypothetical 9.0.2 released with whatever changes we >> think would be useful. When we don’t have big shifts like that, it’s pretty >> easy (I’ve done it more than once actually!).
Re: release blockers for pip
On 8 March 2018 at 03:59, Donald Stufft wrote: > > On Mar 7, 2018, at 12:39 PM, Paul Moore wrote: > > At the moment,we don't have the infrastructure for doing bugfix > releases - and in this specific situation, pulling out the "ready to > go" parts of master to form an interim release isn't really practical, > given the resources we have. Once pip 10 is out of the door, I'd like > to investigate the possibility of having some sort of "maintenance > branch" setup, but we're so thin on the ground at the moment (with > Donald working on Warehouse and Xavier on leave of absence, it's > basically just Pradyun and I, and I'm not managing to actually work on > code much, just reviews and issue management) so I don't want to > overload what little resource we have with admin. > > > > Doing a maintenance release is only a little bit harder than doing a > regular release and I don’t think that maintenance branches fix it. > > If we wanted to we could create a maintenance branch *right now* by just > doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch > off of whatever was released as 9.0.1 that we can cherry-pick changes to. I > don’t think pre-creating this branch at release time adds anything of value > (and in fact I think it makes the situation generally worse). > > * If changes land to master first and then get cherry-picked into a > maintenance branch, then it’s basically no different from what is available > today. > * If changes land to the maintenance branch first, and then get forward > merged to `master`, then people will get confused and send backwards > incompatible changes to the maintenance branch and need to be asked to > rebase their branch onto master. > * Having the branch exist at all will confuse people who don’t know where > to send what branch where. > * In the past, we’ve had bugs get fixed in a maintenance branch, then > forget to merge that into master and “lose” the bug fix. > > Basically, I think sending changes to the maintenance branch first makes > contributing to pip more confusing and more likely we lose things by > accident and sending things to `master` branch then asking for a > cherry-pick to a maintenance branch isn’t really much less effort than > collecting issues at a hypothetical “we want to release 9.0.2” time, > creating a branch then, and cherry-picking them all over at that time. > The one qualifier I'd place on that is that if you did decide to go down the pre-created maintenance branch path, you could probably set up an instance of the cherry-picking bot that Mariatta wrote for CPython, such that the human involvement in the backporting process becomes: - Set the label that tells the maintenance bot to attempt to automatically cherry-pick the squashed-and-merged PR - Review and approve the backport PR (either before or after the CI run completes) - Close the backport PR if the automated backport fails However, if pip can get back to the state of routinely making new releases from master, that will be even simpler to maintain, and the idea of maintenance branches can be deferred until the next large intrusive refactoring candidate is identified. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
Re: release blockers for pip
On 7 March 2018 at 21:25, Jason R. Coombs wrote: > I agree with Donald on prematurely creating branches. A branch is only useful > if it has a commit on it. +1 I wasn't particularly thinking of actual git branches, more whether we needed somewhere for "things that can be released now" vs "things that need some time to complete". On reflection, I prefer an approach where master should be ready to release at all times. Release blocker bugs should be extremely rare (as they imply we put something on master that makes pip basically unusable). > I approach maintenance slightly differently. I try to avoid leaving master > unreleased at any point. Any changes to master get released near-immediately. > The only time a branch is needed is when a back port of a bug fix is needed. I'm quite happy to consider the idea that we increase the pace of releases and work on the basis of master having to always be in a releasable state. Prior to the pip 10 complications, that's essentially how we always worked. Lesson learned, I think :-) > This approach gives maintainers continuous stability and quick, uncomplicated > feedback and simple rollback when something breaks unexpectedly. I think I’ve > only once or twice had to release a bug fix for an older version of a library. > > In this environment, each contrib or batch of accepted contribs creates its > own release, and the scope of that release is determined by semver and the > impact of those changes. It’s minimally branchy, responsive, while giving > users the greatest level of control over which functionality they opt into. > > Without this approach and a mechanical release process, there’s no way I > could maintain 117 packages (thanks to warehouse, I now know that’s how many > I own/maintain), which is why I recommend it. I'd certainly be interested in exploring this option. Once pip 10 is out of the door, I'll look into this in more detail. I'll probably be back with some more questions :-) Thanks, Paul
Re: release blockers for pip
I agree with Donald on prematurely creating branches. A branch is only useful if it has a commit on it. I approach maintenance slightly differently. I try to avoid leaving master unreleased at any point. Any changes to master get released near-immediately. The only time a branch is needed is when a back port of a bug fix is needed. This approach gives maintainers continuous stability and quick, uncomplicated feedback and simple rollback when something breaks unexpectedly. I think I’ve only once or twice had to release a bug fix for an older version of a library. In this environment, each contrib or batch of accepted contribs creates its own release, and the scope of that release is determined by semver and the impact of those changes. It’s minimally branchy, responsive, while giving users the greatest level of control over which functionality they opt into. Without this approach and a mechanical release process, there’s no way I could maintain 117 packages (thanks to warehouse, I now know that’s how many I own/maintain), which is why I recommend it. > On 7 Mar, 2018, at 15:13, Paul Moore wrote: > > Cool, that's good to know. The biggest admin issue I saw was tracking > "what's suitable for a maintenance release", but certainly my > impression is coloured by the big changes that went on since 9.0.1. > > Paul > > On 7 March 2018 at 17:59, Donald Stufft wrote: >> >> On Mar 7, 2018, at 12:39 PM, Paul Moore wrote: >> >> At the moment,we don't have the infrastructure for doing bugfix >> releases - and in this specific situation, pulling out the "ready to >> go" parts of master to form an interim release isn't really practical, >> given the resources we have. Once pip 10 is out of the door, I'd like >> to investigate the possibility of having some sort of "maintenance >> branch" setup, but we're so thin on the ground at the moment (with >> Donald working on Warehouse and Xavier on leave of absence, it's >> basically just Pradyun and I, and I'm not managing to actually work on >> code much, just reviews and issue management) so I don't want to >> overload what little resource we have with admin. >> >> >> >> Doing a maintenance release is only a little bit harder than doing a regular >> release and I don’t think that maintenance branches fix it. >> >> If we wanted to we could create a maintenance branch *right now* by just >> doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch >> off of whatever was released as 9.0.1 that we can cherry-pick changes to. I >> don’t think pre-creating this branch at release time adds anything of value >> (and in fact I think it makes the situation generally worse). >> >> * If changes land to master first and then get cherry-picked into a >> maintenance branch, then it’s basically no different from what is available >> today. >> * If changes land to the maintenance branch first, and then get forward >> merged to `master`, then people will get confused and send backwards >> incompatible changes to the maintenance branch and need to be asked to >> rebase their branch onto master. >> * Having the branch exist at all will confuse people who don’t know where to >> send what branch where. >> * In the past, we’ve had bugs get fixed in a maintenance branch, then forget >> to merge that into master and “lose” the bug fix. >> >> Basically, I think sending changes to the maintenance branch first makes >> contributing to pip more confusing and more likely we lose things by >> accident and sending things to `master` branch then asking for a cherry-pick >> to a maintenance branch isn’t really much less effort than collecting issues >> at a hypothetical “we want to release 9.0.2” time, creating a branch then, >> and cherry-picking them all over at that time. >> >> In either case, a 9.0.2 release is hard because we vastly altered the >> structure of the code between 9.0.1 and `master`, so either solution doesn’t >> really help us get a hypothetical 9.0.2 released with whatever changes we >> think would be useful. When we don’t have big shifts like that, it’s pretty >> easy (I’ve done it more than once actually!).
Re: release blockers for pip
Cool, that's good to know. The biggest admin issue I saw was tracking "what's suitable for a maintenance release", but certainly my impression is coloured by the big changes that went on since 9.0.1. Paul On 7 March 2018 at 17:59, Donald Stufft wrote: > > On Mar 7, 2018, at 12:39 PM, Paul Moore wrote: > > At the moment,we don't have the infrastructure for doing bugfix > releases - and in this specific situation, pulling out the "ready to > go" parts of master to form an interim release isn't really practical, > given the resources we have. Once pip 10 is out of the door, I'd like > to investigate the possibility of having some sort of "maintenance > branch" setup, but we're so thin on the ground at the moment (with > Donald working on Warehouse and Xavier on leave of absence, it's > basically just Pradyun and I, and I'm not managing to actually work on > code much, just reviews and issue management) so I don't want to > overload what little resource we have with admin. > > > > Doing a maintenance release is only a little bit harder than doing a regular > release and I don’t think that maintenance branches fix it. > > If we wanted to we could create a maintenance branch *right now* by just > doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch > off of whatever was released as 9.0.1 that we can cherry-pick changes to. I > don’t think pre-creating this branch at release time adds anything of value > (and in fact I think it makes the situation generally worse). > > * If changes land to master first and then get cherry-picked into a > maintenance branch, then it’s basically no different from what is available > today. > * If changes land to the maintenance branch first, and then get forward > merged to `master`, then people will get confused and send backwards > incompatible changes to the maintenance branch and need to be asked to > rebase their branch onto master. > * Having the branch exist at all will confuse people who don’t know where to > send what branch where. > * In the past, we’ve had bugs get fixed in a maintenance branch, then forget > to merge that into master and “lose” the bug fix. > > Basically, I think sending changes to the maintenance branch first makes > contributing to pip more confusing and more likely we lose things by > accident and sending things to `master` branch then asking for a cherry-pick > to a maintenance branch isn’t really much less effort than collecting issues > at a hypothetical “we want to release 9.0.2” time, creating a branch then, > and cherry-picking them all over at that time. > > In either case, a 9.0.2 release is hard because we vastly altered the > structure of the code between 9.0.1 and `master`, so either solution doesn’t > really help us get a hypothetical 9.0.2 released with whatever changes we > think would be useful. When we don’t have big shifts like that, it’s pretty > easy (I’ve done it more than once actually!).
Re: release blockers for pip
> On Mar 7, 2018, at 12:39 PM, Paul Moore wrote: > > At the moment,we don't have the infrastructure for doing bugfix > releases - and in this specific situation, pulling out the "ready to > go" parts of master to form an interim release isn't really practical, > given the resources we have. Once pip 10 is out of the door, I'd like > to investigate the possibility of having some sort of "maintenance > branch" setup, but we're so thin on the ground at the moment (with > Donald working on Warehouse and Xavier on leave of absence, it's > basically just Pradyun and I, and I'm not managing to actually work on > code much, just reviews and issue management) so I don't want to > overload what little resource we have with admin. Doing a maintenance release is only a little bit harder than doing a regular release and I don’t think that maintenance branches fix it. If we wanted to we could create a maintenance branch *right now* by just doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch off of whatever was released as 9.0.1 that we can cherry-pick changes to. I don’t think pre-creating this branch at release time adds anything of value (and in fact I think it makes the situation generally worse). * If changes land to master first and then get cherry-picked into a maintenance branch, then it’s basically no different from what is available today. * If changes land to the maintenance branch first, and then get forward merged to `master`, then people will get confused and send backwards incompatible changes to the maintenance branch and need to be asked to rebase their branch onto master. * Having the branch exist at all will confuse people who don’t know where to send what branch where. * In the past, we’ve had bugs get fixed in a maintenance branch, then forget to merge that into master and “lose” the bug fix. Basically, I think sending changes to the maintenance branch first makes contributing to pip more confusing and more likely we lose things by accident and sending things to `master` branch then asking for a cherry-pick to a maintenance branch isn’t really much less effort than collecting issues at a hypothetical “we want to release 9.0.2” time, creating a branch then, and cherry-picking them all over at that time. In either case, a 9.0.2 release is hard because we vastly altered the structure of the code between 9.0.1 and `master`, so either solution doesn’t really help us get a hypothetical 9.0.2 released with whatever changes we think would be useful. When we don’t have big shifts like that, it’s pretty easy (I’ve done it more than once actually!).
Re: release blockers for pip
On 7 March 2018 at 16:28, Sumana Harihareswara wrote: > On 03/07/2018 11:22 AM, Sumana Harihareswara wrote: >> I saw today that pip's last release, 9.0.1, was in November 2016. >> https://pypi.org/project/pip/#history >> >> Since that release, 250+ PRs have been merged: >> https://github.com/pypa/pip/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Amerged+updated%3A%3E%3D2016-11-06 >> >> I see that a few issues >> https://github.com/pypa/pip/issues?q=is%3Aopen+is%3Aissue+label%3A%22release+blocker%22 >> are marked as blocking the next release, but are they all blocking *any* >> next release (such as a 9.0.2 bugfix release), or would it be possible >> to release soon anyway, while working towards 10.0.0? Or are there >> backwards-incompatible changes in trunk? >> >> -Sumana > > Of course, a few minutes after posting this, I see Pradyun's roadmap for > releasing 10.0 in > https://github.com/pypa/pip/issues/4981#issuecomment-369495847 . Yeah, we're pushing for a pip 10 release at the moment. I'm currently planning on doing that release, but getting the list of blockers sorted is the current task. I'm hoping to have some time this weekend to go through the list and see where we're up to and update that issue. At the moment,we don't have the infrastructure for doing bugfix releases - and in this specific situation, pulling out the "ready to go" parts of master to form an interim release isn't really practical, given the resources we have. Once pip 10 is out of the door, I'd like to investigate the possibility of having some sort of "maintenance branch" setup, but we're so thin on the ground at the moment (with Donald working on Warehouse and Xavier on leave of absence, it's basically just Pradyun and I, and I'm not managing to actually work on code much, just reviews and issue management) so I don't want to overload what little resource we have with admin. Paul
Re: release blockers for pip
On 03/07/2018 11:22 AM, Sumana Harihareswara wrote: > I saw today that pip's last release, 9.0.1, was in November 2016. > https://pypi.org/project/pip/#history > > Since that release, 250+ PRs have been merged: > https://github.com/pypa/pip/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Amerged+updated%3A%3E%3D2016-11-06 > > I see that a few issues > https://github.com/pypa/pip/issues?q=is%3Aopen+is%3Aissue+label%3A%22release+blocker%22 > are marked as blocking the next release, but are they all blocking *any* > next release (such as a 9.0.2 bugfix release), or would it be possible > to release soon anyway, while working towards 10.0.0? Or are there > backwards-incompatible changes in trunk? > > -Sumana Of course, a few minutes after posting this, I see Pradyun's roadmap for releasing 10.0 in https://github.com/pypa/pip/issues/4981#issuecomment-369495847 . -- Sumana Harihareswara Changeset Consulting https://changeset.nyc
