Re: release blockers for pip

2018-03-19 Thread Sumana Harihareswara
Donald is handling the 9.0.x series. Paul just mentioned in IRC that
he'll be doing the 10.x beta in 2 weeks time.

Pip 9.0.2 is out, and the only change it carries is that it supports
TLSv1.2 when running under system Python on macOS < 10.13. Official
release notes: https://pip.pypa.io/en/stable/news/

Context:
*
https://pyfound.blogspot.com/2017/01/time-to-upgrade-your-python-tls-v12.html
* https://github.com/pypa/warehouse/issues/3293
* https://status.python.org/incidents/btjtz01lzp88


-- 
Sumana Harihareswara
Changeset Consulting
https://changeset.nyc

On 03/07/2018 03:13 PM, Paul Moore wrote:
> Cool, that's good to know. The biggest admin issue I saw was tracking
> "what's suitable for a maintenance release", but certainly my
> impression is coloured by the big changes that went on since 9.0.1.
> 
> Paul
> 
> On 7 March 2018 at 17:59, Donald Stufft  wrote:
>>
>> On Mar 7, 2018, at 12:39 PM, Paul Moore  wrote:
>>
>> At the moment,we don't have the infrastructure for doing bugfix
>> releases - and in this specific situation, pulling out the "ready to
>> go" parts of master to form an interim release isn't really practical,
>> given the resources we have. Once pip 10 is out of the door, I'd like
>> to investigate the possibility of having some sort of "maintenance
>> branch" setup, but we're so thin on the ground at the moment (with
>> Donald working on Warehouse and Xavier on leave of absence, it's
>> basically just Pradyun and I, and I'm not managing to actually work on
>> code much, just reviews and issue management) so I don't want to
>> overload what little resource we have with admin.
>>
>>
>>
>> Doing a maintenance release is only a little bit harder than doing a regular
>> release and I don’t think that maintenance branches fix it.
>>
>> If we wanted to we could create a maintenance branch *right now* by just
>> doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch
>> off of whatever was released as 9.0.1 that we can cherry-pick changes to. I
>> don’t think pre-creating this branch at release time adds anything of value
>> (and in fact I think it makes the situation generally worse).
>>
>> * If changes land to master first and then get cherry-picked into a
>> maintenance branch, then it’s basically no different from what is available
>> today.
>> * If changes land to the maintenance branch first, and then get forward
>> merged to `master`, then people will get confused and send backwards
>> incompatible changes to the maintenance branch and need to be asked to
>> rebase their branch onto master.
>> * Having the branch exist at all will confuse people who don’t know where to
>> send what branch where.
>> * In the past, we’ve had bugs get fixed in a maintenance branch, then forget
>> to merge that into master and “lose” the bug fix.
>>
>> Basically, I think sending changes to the maintenance branch first makes
>> contributing to pip more confusing and more likely we lose things by
>> accident and sending things to `master` branch then asking for a cherry-pick
>> to a maintenance branch isn’t really much less effort than collecting issues
>> at a hypothetical “we want to release 9.0.2” time, creating a branch then,
>> and cherry-picking them all over at that time.
>>
>> In either case, a 9.0.2 release is hard because we vastly altered the
>> structure of the code between 9.0.1 and `master`, so either solution doesn’t
>> really help us get a hypothetical 9.0.2 released with whatever changes we
>> think would be useful. When we don’t have big shifts like that, it’s pretty
>> easy (I’ve done it more than once actually!).



Re: release blockers for pip

2018-03-07 Thread Paul Moore
On 7 March 2018 at 21:25, Jason R. Coombs  wrote:
> I agree with Donald on prematurely creating branches. A branch is only useful 
> if it has a commit on it.

+1 I wasn't particularly thinking of actual git branches, more whether
we needed somewhere for "things that can be released now" vs "things
that need some time to complete". On reflection, I prefer an approach
where master should be ready to release at all times. Release blocker
bugs should be extremely rare (as they imply we put something on
master that makes pip basically unusable).

> I approach maintenance slightly differently. I try to avoid leaving master 
> unreleased at any point. Any changes to master get released near-immediately. 
> The only time a branch is needed is when a back port of a bug fix is needed.

I'm quite happy to consider the idea that we increase the pace of
releases and work on the basis of master having to always be in a
releasable state. Prior to the pip 10 complications, that's
essentially how we always worked. Lesson learned, I think :-)

> This approach gives maintainers continuous stability and quick, uncomplicated 
> feedback and simple rollback when something breaks unexpectedly. I think I’ve 
> only once or twice had to release a bug fix for an older version of a library.
>
> In this environment, each contrib or batch of accepted contribs creates its 
> own release, and the scope of that release is determined by semver and the 
> impact of those changes. It’s minimally branchy, responsive, while giving 
> users the greatest level of control over which functionality they opt into.
>
> Without this approach and a mechanical release process, there’s no way I 
> could maintain 117 packages (thanks to warehouse, I now know that’s how many 
> I own/maintain), which is why I recommend it.

I'd certainly be interested in exploring this option. Once pip 10 is
out of the door, I'll look into this in more detail. I'll probably be
back with some more questions :-)

Thanks,
Paul


Re: release blockers for pip

2018-03-07 Thread Jason R. Coombs
I agree with Donald on prematurely creating branches. A branch is only useful 
if it has a commit on it.

I approach maintenance slightly differently. I try to avoid leaving master 
unreleased at any point. Any changes to master get released near-immediately. 
The only time a branch is needed is when a back port of a bug fix is needed.

This approach gives maintainers continuous stability and quick, uncomplicated 
feedback and simple rollback when something breaks unexpectedly. I think I’ve 
only once or twice had to release a bug fix for an older version of a library.

In this environment, each contrib or batch of accepted contribs creates its own 
release, and the scope of that release is determined by semver and the impact 
of those changes. It’s minimally branchy, responsive, while giving users the 
greatest level of control over which functionality they opt into.

Without this approach and a mechanical release process, there’s no way I could 
maintain 117 packages (thanks to warehouse, I now know that’s how many I 
own/maintain), which is why I recommend it.

> On 7 Mar, 2018, at 15:13, Paul Moore  wrote:
> 
> Cool, that's good to know. The biggest admin issue I saw was tracking
> "what's suitable for a maintenance release", but certainly my
> impression is coloured by the big changes that went on since 9.0.1.
> 
> Paul
> 
> On 7 March 2018 at 17:59, Donald Stufft  wrote:
>> 
>> On Mar 7, 2018, at 12:39 PM, Paul Moore  wrote:
>> 
>> At the moment,we don't have the infrastructure for doing bugfix
>> releases - and in this specific situation, pulling out the "ready to
>> go" parts of master to form an interim release isn't really practical,
>> given the resources we have. Once pip 10 is out of the door, I'd like
>> to investigate the possibility of having some sort of "maintenance
>> branch" setup, but we're so thin on the ground at the moment (with
>> Donald working on Warehouse and Xavier on leave of absence, it's
>> basically just Pradyun and I, and I'm not managing to actually work on
>> code much, just reviews and issue management) so I don't want to
>> overload what little resource we have with admin.
>> 
>> 
>> 
>> Doing a maintenance release is only a little bit harder than doing a regular
>> release and I don’t think that maintenance branches fix it.
>> 
>> If we wanted to we could create a maintenance branch *right now* by just
>> doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch
>> off of whatever was released as 9.0.1 that we can cherry-pick changes to. I
>> don’t think pre-creating this branch at release time adds anything of value
>> (and in fact I think it makes the situation generally worse).
>> 
>> * If changes land to master first and then get cherry-picked into a
>> maintenance branch, then it’s basically no different from what is available
>> today.
>> * If changes land to the maintenance branch first, and then get forward
>> merged to `master`, then people will get confused and send backwards
>> incompatible changes to the maintenance branch and need to be asked to
>> rebase their branch onto master.
>> * Having the branch exist at all will confuse people who don’t know where to
>> send what branch where.
>> * In the past, we’ve had bugs get fixed in a maintenance branch, then forget
>> to merge that into master and “lose” the bug fix.
>> 
>> Basically, I think sending changes to the maintenance branch first makes
>> contributing to pip more confusing and more likely we lose things by
>> accident and sending things to `master` branch then asking for a cherry-pick
>> to a maintenance branch isn’t really much less effort than collecting issues
>> at a hypothetical “we want to release 9.0.2” time, creating a branch then,
>> and cherry-picking them all over at that time.
>> 
>> In either case, a 9.0.2 release is hard because we vastly altered the
>> structure of the code between 9.0.1 and `master`, so either solution doesn’t
>> really help us get a hypothetical 9.0.2 released with whatever changes we
>> think would be useful. When we don’t have big shifts like that, it’s pretty
>> easy (I’ve done it more than once actually!).



Re: release blockers for pip

2018-03-07 Thread Paul Moore
Cool, that's good to know. The biggest admin issue I saw was tracking
"what's suitable for a maintenance release", but certainly my
impression is coloured by the big changes that went on since 9.0.1.

Paul

On 7 March 2018 at 17:59, Donald Stufft  wrote:
>
> On Mar 7, 2018, at 12:39 PM, Paul Moore  wrote:
>
> At the moment,we don't have the infrastructure for doing bugfix
> releases - and in this specific situation, pulling out the "ready to
> go" parts of master to form an interim release isn't really practical,
> given the resources we have. Once pip 10 is out of the door, I'd like
> to investigate the possibility of having some sort of "maintenance
> branch" setup, but we're so thin on the ground at the moment (with
> Donald working on Warehouse and Xavier on leave of absence, it's
> basically just Pradyun and I, and I'm not managing to actually work on
> code much, just reviews and issue management) so I don't want to
> overload what little resource we have with admin.
>
>
>
> Doing a maintenance release is only a little bit harder than doing a regular
> release and I don’t think that maintenance branches fix it.
>
> If we wanted to we could create a maintenance branch *right now* by just
> doing ``git checkout -b release/9.0.2 9.0.1`` which would create a branch
> off of whatever was released as 9.0.1 that we can cherry-pick changes to. I
> don’t think pre-creating this branch at release time adds anything of value
> (and in fact I think it makes the situation generally worse).
>
> * If changes land to master first and then get cherry-picked into a
> maintenance branch, then it’s basically no different from what is available
> today.
> * If changes land to the maintenance branch first, and then get forward
> merged to `master`, then people will get confused and send backwards
> incompatible changes to the maintenance branch and need to be asked to
> rebase their branch onto master.
> * Having the branch exist at all will confuse people who don’t know where to
> send what branch where.
> * In the past, we’ve had bugs get fixed in a maintenance branch, then forget
> to merge that into master and “lose” the bug fix.
>
> Basically, I think sending changes to the maintenance branch first makes
> contributing to pip more confusing and more likely we lose things by
> accident and sending things to `master` branch then asking for a cherry-pick
> to a maintenance branch isn’t really much less effort than collecting issues
> at a hypothetical “we want to release 9.0.2” time, creating a branch then,
> and cherry-picking them all over at that time.
>
> In either case, a 9.0.2 release is hard because we vastly altered the
> structure of the code between 9.0.1 and `master`, so either solution doesn’t
> really help us get a hypothetical 9.0.2 released with whatever changes we
> think would be useful. When we don’t have big shifts like that, it’s pretty
> easy (I’ve done it more than once actually!).


Re: release blockers for pip

2018-03-07 Thread Donald Stufft

> On Mar 7, 2018, at 12:39 PM, Paul Moore  wrote:
> 
> At the moment,we don't have the infrastructure for doing bugfix
> releases - and in this specific situation, pulling out the "ready to
> go" parts of master to form an interim release isn't really practical,
> given the resources we have. Once pip 10 is out of the door, I'd like
> to investigate the possibility of having some sort of "maintenance
> branch" setup, but we're so thin on the ground at the moment (with
> Donald working on Warehouse and Xavier on leave of absence, it's
> basically just Pradyun and I, and I'm not managing to actually work on
> code much, just reviews and issue management) so I don't want to
> overload what little resource we have with admin.


Doing a maintenance release is only a little bit harder than doing a regular 
release and I don’t think that maintenance branches fix it.

If we wanted to we could create a maintenance branch *right now* by just doing 
``git checkout -b release/9.0.2 9.0.1`` which would create a branch off of 
whatever was released as 9.0.1 that we can cherry-pick changes to. I don’t 
think pre-creating this branch at release time adds anything of value (and in 
fact I think it makes the situation generally worse).

* If changes land to master first and then get cherry-picked into a maintenance 
branch, then it’s basically no different from what is available today.
* If changes land to the maintenance branch first, and then get forward merged 
to `master`, then people will get confused and send backwards incompatible 
changes to the maintenance branch and need to be asked to rebase their branch 
onto master.
* Having the branch exist at all will confuse people who don’t know where to 
send what branch where.
* In the past, we’ve had bugs get fixed in a maintenance branch, then forget to 
merge that into master and “lose” the bug fix.

Basically, I think sending changes to the maintenance branch first makes 
contributing to pip more confusing and more likely we lose things by accident 
and sending things to `master` branch then asking for a cherry-pick to a 
maintenance branch isn’t really much less effort than collecting issues at a 
hypothetical “we want to release 9.0.2” time, creating a branch then, and 
cherry-picking them all over at that time.

In either case, a 9.0.2 release is hard because we vastly altered the structure 
of the code between 9.0.1 and `master`, so either solution doesn’t really help 
us get a hypothetical 9.0.2 released with whatever changes we think would be 
useful. When we don’t have big shifts like that, it’s pretty easy (I’ve done it 
more than once actually!).

Re: release blockers for pip

2018-03-07 Thread Paul Moore
On 7 March 2018 at 16:28, Sumana Harihareswara  wrote:
> On 03/07/2018 11:22 AM, Sumana Harihareswara wrote:
>> I saw today that pip's last release, 9.0.1, was in November 2016.
>> https://pypi.org/project/pip/#history
>>
>> Since that release, 250+ PRs have been merged:
>> https://github.com/pypa/pip/pulls?utf8=%E2%9C%93=is%3Apr+is%3Amerged+updated%3A%3E%3D2016-11-06
>>
>> I see that a few issues
>> https://github.com/pypa/pip/issues?q=is%3Aopen+is%3Aissue+label%3A%22release+blocker%22
>> are marked as blocking the next release, but are they all blocking *any*
>> next release (such as a 9.0.2 bugfix release), or would it be possible
>> to release soon anyway, while working towards 10.0.0? Or are there
>> backwards-incompatible changes in trunk?
>>
>> -Sumana
>
> Of course, a few minutes after posting this, I see Pradyun's roadmap for
> releasing 10.0 in
> https://github.com/pypa/pip/issues/4981#issuecomment-369495847 .

Yeah, we're pushing for a pip 10 release at the moment. I'm currently
planning on doing that release, but getting the list of blockers
sorted is the current task. I'm hoping to have some time this weekend
to go through the list and see where we're up to and update that
issue.

At the moment,we don't have the infrastructure for doing bugfix
releases - and in this specific situation, pulling out the "ready to
go" parts of master to form an interim release isn't really practical,
given the resources we have. Once pip 10 is out of the door, I'd like
to investigate the possibility of having some sort of "maintenance
branch" setup, but we're so thin on the ground at the moment (with
Donald working on Warehouse and Xavier on leave of absence, it's
basically just Pradyun and I, and I'm not managing to actually work on
code much, just reviews and issue management) so I don't want to
overload what little resource we have with admin.

Paul


Re: release blockers for pip

2018-03-07 Thread Sumana Harihareswara
On 03/07/2018 11:22 AM, Sumana Harihareswara wrote:
> I saw today that pip's last release, 9.0.1, was in November 2016.
> https://pypi.org/project/pip/#history
> 
> Since that release, 250+ PRs have been merged:
> https://github.com/pypa/pip/pulls?utf8=%E2%9C%93=is%3Apr+is%3Amerged+updated%3A%3E%3D2016-11-06
> 
> I see that a few issues
> https://github.com/pypa/pip/issues?q=is%3Aopen+is%3Aissue+label%3A%22release+blocker%22
> are marked as blocking the next release, but are they all blocking *any*
> next release (such as a 9.0.2 bugfix release), or would it be possible
> to release soon anyway, while working towards 10.0.0? Or are there
> backwards-incompatible changes in trunk?
> 
> -Sumana

Of course, a few minutes after posting this, I see Pradyun's roadmap for
releasing 10.0 in
https://github.com/pypa/pip/issues/4981#issuecomment-369495847 .

-- 
Sumana Harihareswara
Changeset Consulting
https://changeset.nyc