On May 16, 2014, at 3:12 PM, holger krekel <[email protected]> wrote:
> On Fri, May 16, 2014 at 13:38 -0500, Carl Meyer wrote: >> On 05/16/2014 12:10 PM, Donald Stufft wrote: >>>> 2. Add a deprecation path for --allow-unverified; can describe it in >>>> general terms as "the PEP 438 installer flag allowing installation of >>>> unverified external packages" if you don't want to be pip-specific. >>>> Currently PEP 470 has no mention of this, but I think letting a >>>> deprecation of --allow-unverified fully run its course _before_ making >>>> breaking changes on the PyPI side is a critical part of making this >>>> transition in a user-friendlier way. >>> >>> Perhaps I wasn't being as obvious as I thought! My goal was to nail down >>> what >>> the final destination looked like, and then once we figured out an end goal >>> deprecate everything that doesn't match that end goal (and ultimately remove >>> it). >>> >>> One thing I don't want to do is add another intermediary fix with it's own >>> combination of flags that will do the thing that the user wants to do. We >>> already have: >>> >>> 1) No additional flags >>> 2) --allow-external + --allow-insecure >>> 3) --allow-external + --allow-unverified >>> 4) --allow-unverified >>> >>> Depending on what version of pip you're using[1]. I really really want the >>> 5th >>> incantation of this to be the final incantation. That was one of the reasons >>> why I went the way I did in PEP 470. I don't believe it's possible to move >>> away from these without a break in compatibility for <=7% of projects, >>> *however* a key benefit of PEP 470 is that the mechanisms for allowing >>> additional locations has existed in pip for a long time. We can have a >>> singular >>> clear message that says "If you want to do X then use these flags" and it >>> doesn't matter what version you're on. I vastly prefer that to the current >>> situation (and the "just let the deprecation run it's course" proposal) >>> where >>> you have to pick the right combination of flags based on pip version. >> >> I guess the key thing I don't understand yet is, why would we assume >> that any package that hasn't already switched to verified-external-links >> since PEP 438, given a one-year window so far to do so, is any more >> likely to populate this new discoverable-index metadata from PEP 470, >> given a six-month window? > > That's a crucial question indeed (and i also agree to your earlier summary > of the situation, btw). > >> It seems like PEP 470 places a lot of weight on an assumption of active >> maintainers, when really the core problem is a significant chunk of >> packages that (from the evidence of PEP 438 uptake) don't have active >> maintainers. >> >> So if we conclude that the bulk of the problematic legacy packages will >> probably never populate this new discoverable-index metadata either, at >> what stage in the process would their users get any useful clue as to >> how to continue to install that package? >> >> One option is Holger's solution: scraping the current links and >> populating them as verified external links. We both don't like this >> because it involves PyPI misleading users about the status of those >> links, and you also don't like it because you want to deprecate verified >> external links too. > > Regarding the last bit, i indeed don't want to see the possibility removed > to register external links-with-checksum. PEP438 just introduced it and > i think it's sufficient and efficient for allowing maintainers to host > files externally while making it easy for end users choose to accept it > along with a slight loss of reliability that "--allow-externals" might > entail. Integrity guruantees of such external links and pypi-uploaded > files are strictly the same. If i understand correctly, PEP470 aims to > discontinue this PEP438 mechanism and i strongly disagree with that. This would discontinue that mechanism. It has proven to be confusing and toxic and practically unused. It’s also not entirely true that it’s new with PEP438. It existed prior to that for a long time you just had to put a hashed url in your download_url. > >> Another option is if the deprecation message for --allow-unverified also >> gives the user the exact --find-links URL(s) they need to install that >> same package/version (which I think is implementable in pip today >> without any changes in PyPI). The downside here is that it really >> doesn't improve the current UX for these legacy packages, it just >> replaces --allow-unverified with explicit --find-links: but at least the >> latter is more explicit and places the responsibility clearly on the >> external host, not PyPI. > > It also places the responsibility to have project-specific options > on the end user. I prefer this path of action to the current PEP470, however. > >> Or, thirdly, Paul's proposal could solve this, if PyPI automatically >> generated an "external legacy index" for any packages that haven't >> generated their own external index URL by a certain date. Really in a >> way this is similar to Holger's proposal, except it uses >> external-indexes instead of verified-external-URLs, and is again a bit >> more explicit about what's going on (at the cost of requiring more >> adjustment from users). > > I find Paul's idea most interesting. What about this pypi-hosted > legacy index providing "external" links so that you would still need to > say "--allow-external"? I am imagining something like: > > pip install --extra-index https://pypi-stale-unsafe.python.org/ > --allow-external I do not want —allow-external to remain. However I’m still thinking Paul’s idea over. If it was done it would not require the —allow-external parameter. > > This index could be a statically served nginx site because > nothing would be added to it dynamically by users. It would > contain all of the "auto-crawled" packages with the snapshot > checksums we found at the time. And the "--allow-external" > signals that the files are not hosted on pypi. No maintainer would > need to setup anything for his project and end users could add > this one extra index if they somehow depend on those packages (and > get the hint from newer pip versions and from the PyPI main page). > > Going down this route would allow to get rid of "--allow-insecure", > "--allow-unverified" and we'd only have the PEP438 "pypi-explicit" > hosting mode. > > best, > holger > >> Basically, I think some acknowledgment of this problem of packages >> without active maintainers (and ideally a proposed solution to it) >> should be in PEP 470. > > >> Carl >> > > > >> _______________________________________________ >> Distutils-SIG maillist - [email protected] >> https://mail.python.org/mailman/listinfo/distutils-sig ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
