FYI pipenv have been vendoring pip since pretty early in the projects existence, since before 10 was released. At no point was I in touch to say that was a major issue though because we use the api precisely as you describe, *at our own risk*. Every time I’ve been in touch it’s been to ask how to go about starting the conversation around sharing elements of a codebase, to which the answer has universally been ‘don’t use our internals, go write a library if you want’. So as a result we’ve mostly gone away and kept pip vendored as we started some libraries.
The internal changes didn’t do much as far as I’m concerned, and as we knew the risks and had a vendored copy we were able to upgrade on our own clock. We use this copy only to fetch things from the index for generating lockfiles. Also there was significant publicity saying that people shouldn’t use internals and it seemed like it should honestly not be on you to worry about — that’s the whole point of vendoring it. I don’t think any of us saw a point in trying to force that problem on pip given the internals changes. And pipenv was not a part of the pypa at the start of the project, so we certainly _were_ working in isolation at that time. To Chris’ broader point it is definitely duplicated effort and I am in full agreement which is why I want to establish which code should be extracted and generalized and where it should be maintained. But as Paul mentioned there also is no PyPA strategy at play here. We aren’t trying to move into pip’s space and vice versa AFAIU but I’m also not too sure how much trying to point a finger at us here is helpful. You find it frustrating that we are working in isolation, I find it frustrating that I keep being pushed to go away and write libraries whenever I mention sharing functionality, which to me sounds like you want me to work in isolation. Obviously there is some communication breakdown— At the end of the day I just want to give users a consistent experience. Hopefully this is a goal others share. As far as the pipenv roadmap, we are rewriting a good chunk of it but it essentially the atomic things we do or would like to do with regards to shared functionality at a high level: - parse user input/requirements - retrieve package metadata - resolve dependencies (not in pip right now obviously) — may involve building wheels/sdists - download / build packages - install packages (into a virtualenv sometimes) - uninstall packages (from a virtualenv sometimes) - list packages etc (in a virtualenv sometimes) We know how these are accomplished in pip and we know how we are handling them both in pipenv and in our alternative implementations, which I believe still have some dependency on pip. For us the roadmap is basically to figure out - where do the abstractions live - how do we start - ??? Dan Ryan // pipenv maintainer gh: @techalchemy > On Sep 25, 2018, at 8:11 AM, Paul Moore <p.f.mo...@gmail.com> wrote: > >> On Tue, 25 Sep 2018 at 12:53, Chris Jerdonek <chris.jerdo...@gmail.com> >> wrote: >> >>> On Tue, Sep 25, 2018 at 3:47 AM, Tzu-ping Chung <uranu...@gmail.com> wrote: >>> We are using pip internals for things pip wasn’t implemented for. >>> Specifically, >>> Pipenv uses pip’s package-fetching functions to implement its >>> platform-agnostic >>> resolver. pip does not have this, so there’s no functional overlap here. >>> Those >>> utilities are used to build something that doesn’t exist in pip, so there’s >>> no >>> duplicated efforts. >>> >>> My recent focus on making sense of packaging implementations and splitting >>> out >>> parts of pip is exactly to prevent potential duplicated efforts. If we >>> can’t use pip >>> internals, let’s make things we want to use *not* internal! >> >> I don't see this practice of "splitting out parts of pip" actually >> being followed so far though, and I want to tell you how it's leading >> to duplicated efforts right now -- even for the PR's I happen to be >> working on today. > > [...] > >> So by creating implementations of functions separate from pip instead >> of splitting them out of pip, it's leading to duplicated work for >> people working on pip's code base. A process of splitting something >> out of pip would I think involve a process more like what I'm >> following -- namely to refactor functionality in pip *first* so it can >> be used separately, and then pull that out so you only have one >> implementation at any point in time. Otherwise, you wind up creating >> two versions of similar functionality that not only need to be created >> twice, but also later reconciled if the plan is merge them back >> together. > > This is a very good point, and thanks for the concrete example. For > the record, I'm also a little annoyed that pipenv have been doing all > this work in isolation. There's been very little communication between > the pip and pipenv projects, and I think that's contributed to the > problem. I, for one, wasn't even aware for a long time that they were > embedding their own copy of pip. When we made pip's internals in > private in pip 10, we got essentially no feedback from anyone saying > that it would be a problem, and finding out that it had a sufficiently > major impact on pipenv that they had to embed the old version of pip > *after* the release was (to put it politely) suboptimal :-( > > Having said all this, this thread is the start of an attempt to work > better together and I think we should be glad that it's happening and > try to make it work. But I don't think we've got off to a particularly > good start and it will need work. In particular, I'd like to see an > attempt from *both* projects to clarify our goals - I've been speaking > solely for myself so far, it would be good to get the other pip > developers' views and set out a sort of roadmap, and for pipenv to do > the same. > > Paul > -- > Distutils-SIG mailing list -- distutils-sig@python.org > To unsubscribe send an email to distutils-sig-le...@python.org > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/OGXUNDRBLTNPOWLLFBVF45SW4SRGYSPP/ -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/T5LB6IRACQDMMJZC6N5LMQGWQ3I63OAU/
