Hi again, On Fri, Apr 6, 2018 at 7:21 PM, Donald Stufft <don...@stufft.io> wrote: > > > On Apr 6, 2018, at 12:30 PM, Matthew Brett <matthew.br...@gmail.com> wrote: > > Hi, > > On Fri, Apr 6, 2018 at 4:49 PM, Donald Stufft <don...@stufft.io> wrote: > > > On Apr 6, 2018, at 11:25 AM, Matthew Brett <matthew.br...@gmail.com> wrote: > > > As of 8th April, pip will break, largely in silence. As used with > default command line options, pip will stop seeing anything on pypi. > It will tell the user that it can't find packages that do exist on > pypi, that that users are up to date for packages that have later > versions on pypi - including pip. There are no messages to warn the > user what has happened or how to fix it. It is easy to think you've > upgraded to the latest version, when you have not. > > > This isn’t quite accurate: > > This is not a blanket issue for anyone on macOS < 10.13. It *only* affects > people using a Python that links against the ancient OpenSSL provided by > Apple, of which the #1 cause is System Python on macOS < 10.13, however it > is also the 2.7 Python.org macOS installer on any version of Python. All > other forms of getting Python, to my knowledge, link against a newer version > of OpenSSL (or against LibreSSL in System Python on macOS 10.13).  > > > I think that list is for current installers - then there's the tail of > older installs, of which I am one (a Python.org 3.5 install). Only > relevant, because this is going to be a complex patchwork of affected > users. > > This affects a small percentage of users, Of the downloads from PyPI in the > last 7 days that originated from a macOS system, 6% of them would have > failed, 94% of them would have succeeded. > > > 6% of very confused users is a lot. But - I guess some proportion of > that 100% is continuous integration installs? These will tend to > have very up to date Pythons. Is there any way to estimate how many > non-bot users will be affected? > > > Not that I am aware of. Note— that is 6% of downloads, not 6% of users, that > could have been a whole lot users each downloading a single package, or one > user downloading thousands, we can’t tell. > > > Once we are at 100% unavailability, we can ask our CDN to disable TLSv1.0 > and TLSv1.1 completely, which will raise an OpenSSL error message instead of > a HTTP error message. That will look like (ignore the index-url, used that > because it already has TLSv1.0/1.1 disabled completely): > > $ /usr/local/bin/python2.7 -m pip install --index-url > https://files.pythonhosted.org/ --upgrade pip > Could not fetch URL https://files.pythonhosted.org/pip/: There was a > problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] > tlsv1 alert protocol version (_ssl.c:661) - skipping > Requirement already up-to-date: pip in > /Library/Frameworks/Python.framework/Versions/2.7.14_10_6/lib/python2.7/site-packages > You are using pip version 9.0.1, however version 9.0.3 is available. > You should consider upgrading via the 'pip install --upgrade pip' > command. > > > Yes - that's better than the current silent break, but still it's a > shame that the message is so obscure given the long lead-in. > > Finally, even If the above weren’t true, there really isn’t much we can do > either way. We can’t go back in time and change the already released version > of pip, and we only have so many ways that PyPI can signal an error to pip > one is using an HTTP status code (what the brownouts do now) and one is > rejecting the connection with a TLS error (what the above snippet does). > Beyond that, any improvements require getting users to upgrade their version > of pip, and if we can get them to upgrade for a better error message, > presumably we can get them to upgrade for the SecureTransport fallback in > 9.0.3+. > > > I was hoping against hope that there was another way of generating an > informative error from the server that would force itself on the > user's attention, perhaps by subverting another mechanism. Is there > really no way for pypi to transmit an informative error when it > detects a request from an affected system? > > > No, there’s not. pip makes HTTP requests and there’s no place for extra > metadata attached those requests except in the HTTP status code (which as > you noted, pip swallows by default because historically we didn’t know if > the URL was expected to work or not). The simple API wasn’t really designed, > it evolved out of the primordial ooze.
Ouch - it does look like we're in a very difficult situation, which might apply to a significant number of users. I think the worst option is what we have ramping up in the brownout, which is the silent failure to upgrade or find packages. Yes, you get something informative with the -v option if you think to try it, but I think the confusion from the not -v state is of much greater harm than the help from the -v message. The better option, I believe, is just to go straight to the SSL error. At least that will set people Googling. They might be annoyed, but at least they won't be mystified. Ideally, that would happen as late as possible, to give people the maximum chance to upgrade with the usual pip upgrade commands. One of the nasty features of this one is that it breaks the normal pip upgrade method. Can we push back the SSL error until it's forced upon us in June? Cheers, Matthew