On Tue, Sep 25, 2018 at 3:47 AM, Tzu-ping Chung <uranu...@gmail.com> wrote: > We are using pip internals for things pip wasn’t implemented for. > Specifically, > Pipenv uses pip’s package-fetching functions to implement its > platform-agnostic > resolver. pip does not have this, so there’s no functional overlap here. Those > utilities are used to build something that doesn’t exist in pip, so there’s no > duplicated efforts. > > My recent focus on making sense of packaging implementations and splitting out > parts of pip is exactly to prevent potential duplicated efforts. If we can’t > use pip > internals, let’s make things we want to use *not* internal!
I don't see this practice of "splitting out parts of pip" actually being followed so far though, and I want to tell you how it's leading to duplicated efforts right now -- even for the PR's I happen to be working on today. Earlier in this thread, Dan said the pipenv maintainers are reimplementing certain pip functionality rather than splitting it out: "we basically had to start by calling pip directly, then slowly reimplement each aspect of the underlying logic using various elements in distlib/setuptools or rebuilding those." And then he listed seven libraries they are working on that contain code like this. Today I happened to file a simple PR to pip to continue DRYing up pip's VCS-related code base and making it more testable, etc: https://github.com/pypa/pip/pull/5810 My PR adds a function called `make_vcs_requirement_url()` and refactors various parts of pip to use it: https://github.com/pypa/pip/pull/5810/files#diff-c9e9f4633cef78355bd4a581febff330R859 However, when I look at requirementslib (one of the libraries Dan mentioned), I see they have already reimplemented the function I added in my PR. In requirementslib, it's called `build_vcs_link()`: https://github.com/sarugaku/requirementslib/blob/d457054d969fcb5e712c8f74f9ddac182335443b/src/requirementslib/models/utils.py#L79-L95 So instead of splitting it out of pip and modifying pip to use that, they created a new implementation separate from pip. And now I am redoing the implementation a second time. Other VCS-related functions in that same requirementslib file (e.g. "add_ssh_scheme_to_git_uri()") overlap with VCS code in pip that I've been doing similar work on. So by creating implementations of functions separate from pip instead of splitting them out of pip, it's leading to duplicated work for people working on pip's code base. A process of splitting something out of pip would I think involve a process more like what I'm following -- namely to refactor functionality in pip *first* so it can be used separately, and then pull that out so you only have one implementation at any point in time. Otherwise, you wind up creating two versions of similar functionality that not only need to be created twice, but also later reconciled if the plan is merge them back together. --Chris > > TP > > >> On 25/9, 2018, at 18:38, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: >> >> On Tue, Sep 25, 2018 at 3:21 AM, Nick Coghlan <ncogh...@gmail.com> wrote: >>> On Tue, 25 Sep 2018 at 19:48, Chris Jerdonek <chris.jerdo...@gmail.com> >>> wrote: >>>> What I'm trying to gauge is, if the plan is for pipenv not to depend >>>> on pip, and pipenv has strictly greater functionality than pip, then >>>> what purpose will PyPA have in continuing to develop pip in addition >>>> to pipenv? >>> >>> That's not the plan, as all of pip's features for actually >>> installing/uninstalling packages, and for introspecting the *as built* >>> package environment, aren't things where pipenv's needs diverge from >>> pip's. >> >> That's not what Tzu-ping said though. In an earlier email, he said, >> "If we can’t use pip internals, then yes, the plan is to not depend on >> pip." >> >> --Chris >> >> >>> >>> Where their needs diverge is at the dependency resolver level, as >>> pipenv needs to be able to generate a lock file for an arbitrary >>> target environment that may not match the currently running Python >>> interpreter *without* necessarily installing those packages anywhere >>> (although it may need to build wheels to get the dependency listing), >>> whereas pip has the more concrete task of "get theses packages and >>> their dependencies installed into the currently active environment". >>> >>> If it helps, think of pipenv relating to pip in much the same way as >>> pip-tools (pip-compile/pip-sync) relates to pip, just with Pipfile and >>> Pipfile.lock instead of requirements.in and requirements.txt. >>> >>> Cheers, >>> Nick. >>> >>> -- >>> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/ERR2YSW3VWYAQCQ4BHPRWOGTDVICVYUH/
