Pipenv’s resolver does not use troove classifiers as far as I am aware. I could be missing something though, the current resolver implementation is not the best code base to reason with.
Can you provide some ideas on this, Dan? TP > On 20/9, 2018, at 15:09, Bernat Gabor <gaborjber...@gmail.com> wrote: > > One of such differences is pipenv apparently trying to work from the troove > classifiers for version compatibility instead of the existing python_requires > (see https://github.com/tox-dev/tox/pull/1005 > <https://github.com/tox-dev/tox/pull/1005>). Why? Point in case how people > start to think pipenv is pip. At this point, pip is a completely different > beast than pip I think. > > Such check at best would fall under the ``twine check`` command at PyPi > upload time, not at install time. > > On Thu, Sep 20, 2018 at 8:03 AM Tzu-ping Chung <uranu...@gmail.com > <mailto:uranu...@gmail.com>> wrote: > > >> On 20/9, 2018, at 13:22, Chris Jerdonek <chris.jerdo...@gmail.com >> <mailto:chris.jerdo...@gmail.com>> wrote: >> >> On Wed, Sep 19, 2018 at 8:54 PM, Dan Ryan <d...@danryan.co >> <mailto:d...@danryan.co>> wrote: >> I should clarify that we have already implemented a number of these as >> libraries over the last several months (and I am super familiar with pip's >> internals by now and I'm sure TP is getting there as well). More on this >> below >> ... >> We are super cognizant of that aspect as I am pretty sure we are hitting >> this wall in a full (nearly) pip-free reimplementation of all of the pipenv >> internals from the ground up, including wheel building/installation, but we >> basically had to start by calling pip directly, then slowly reimplement each >> aspect of the underlying logic using various elements in distlib/setuptools >> or rebuilding those. >> >> Is the hope or game plan then for pipenv not to have to depend on pip? This >> is partly what I was trying to learn in my email to this list a month ago >> (on Aug. 20, with subject: "pipenv and pip"): >> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/thread/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/ >> >> <https://mail.python.org/mm3/archives/list/distutils-sig@python.org/thread/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/> >> >> Based on the replies, I wasn't getting that impression at the time (though I >> don't remember getting a clear answer), but maybe things have changed since >> then. > > The resolution side of Pipenv really needs a Python API, and also cannot > really > use the CLI because it needs something slightly different than pip’s > high-level > logic (Nick mentioned this briefly). If we can’t use pip internals, then yes, > the plan > is to not depend on pip. The hope is we can share those internals with pip > (either > following the same standards, or using the same implementation), hence my > series > of questions. > > The installation side of Pipenv will continue to use pip directly, at least > for a while > more even after the resolution side breaks away, since “pip install” is > adequate > enough for our purposes. There are some possible improvements if there is a > lower-layer library (e.g. to avoid pip startup overhead), but that is far > less important. > > >> >> It should certainly be a lot easier for pipenv to move fast since there is >> no legacy base of users to maintain compatibility with. However, I worry >> about the fracturing this will cause. In creating these libraries, from the >> pip tracker it doesn't look like any effort is going into refactoring pip to >> make use of them. This relates to the point I made earlier today about how >> there won't be an easy way to cut pip over to using a new library unless an >> effort is made from the beginning. Thus, it's looking like things could be >> on track to split the user and maintainer base in two, with pip bearing the >> legacy burden and perhaps not seeing the improvements. Are we okay with that >> future? > > I’m afraid the new implementation will still need to deal with compatibility > issues. > Users expect Pipenv to work exactly as pip, and get very angry if it does not, > especially when they see it is under the PyPA organisation on GitHub. The last > time Pipenv tries to explain it does whatever arbitrary things it does, we get > labelled as “toxic” (there are other issues in play, but this is IMO the > ultimate > cause). Whether the image is fair or not, I would most definitely want to > avoid > similar incidents from happening again. > > I think Pipenv would be okay to maintain a different (from scratch) > implementation > than pip’s, but it would still need to do (almost) exactly what pip is doing, > unless > we can have people (pip maintainers or otherwise) backing the differences. > Whether > pip uses the new implementation or not wouldn’t change the requirement :( > > > TP > > >> >> --Chris >> >> >> Since you mentioned following along, here's what we're working on right now: >> >> https://github.com/sarugaku/requirementslib >> <https://github.com/sarugaku/requirementslib> -- abstraction layer for >> parsing >> and converting various requirements formats >> (pipfile/requirements.txt/command line/InstallRequirement) and moving >> between all of them >> https://github.com/sarugaku/resolvelib >> <https://github.com/sarugaku/resolvelib> -- directed acyclic graph library >> for >> handling dependency resolution (not yet being used in pipenv) >> https://github.com/sarugaku/passa <https://github.com/sarugaku/passa> -- >> dependency resolver/installer/pipfile >> manager (bulk of the logic we have been talking about is in here right now) >> -- I think we will probably split this back out into multiple other smaller >> libraries or something based on the discussion >> https://github.com/sarugaku/plette <https://github.com/sarugaku/plette> -- >> this is a rewrite of pipfile with some >> additional logic / validation >> https://github.com/sarugaku/shellingham >> <https://github.com/sarugaku/shellingham> -- this is a shell detection >> library >> made up of some tooling we built in pipenv for environment detection >> https://github.com/sarugaku/pythonfinder >> <https://github.com/sarugaku/pythonfinder> -- this is a library for finding >> python (pep 514 compliant) by version and for finding any other executables >> (cross platform) >> https://github.com/sarugaku/virtenv <https://github.com/sarugaku/virtenv> -- >> python api for virtualenv creation >> >> Happy to provide access or take advice as needed on any of those. Thanks >> all for the receptiveness and collaboration >> >> Dan Ryan >> gh: @techalchemy // e: d...@danryan.co <mailto:d...@danryan.co> >> >> From: Donald Stufft [mailto:don...@stufft.io <mailto:don...@stufft.io>] >> Sent: Wednesday, September 19, 2018 1:52 PM >> To: Tzu-ping Chung >> Cc: Distutils >> Subject: [Distutils] Re: Distlib vs Packaging (Was: disable building wheel >> for a package) >> >> My general recommendation if you want a Python implementation/interface for >> something pip does, is: >> >> - Open an issue on the pip repository to document your intent and to make >> sure that there is nobody there who is against having that functionality >> split out. This might also give a chance for people with familiarity in that >> API to mention pain points that you can solve in a new API. We can also >> probably give you a good sense if the thing you want in a library is >> something that probably has multiple things that are dependent on getting >> split out first (for instance, if you said you wanted a library for >> installing wheels, we'd probably tell you that there is a dependency on PEP >> 425 tags, pip locations, maybe other that need resolved first) and also >> whether this is something that should have a PEP first or not. Getting some >> rough agreement on the plan to split X thing out before you start is overall >> a good thing. >> >> - Create or update a PEP if required, and get it into the provisional state. >> >> - Make the library, either as a PR to packaging or as it's own independent >> library. If there are questions that come up while creating that library/PR >> that have to do with specific pip behaviors, go back to that original issue >> and ask for clarification etc. Ideally at some point you'll open a PR on pip >> that uses the new library (my suggestion is to not bundle the library in the >> initial PR, and just import it normally so that the PR diff doesn't include >> the full bundled library until there's agreement on it). If there's another >> tool (pipenv, whatever) that is looking to use that same functionality, open >> a WIP PR there too that switches it to using that. Use feedback and what you >> learn from trying to integrate in those libraries to influence back the >> design of the API itself. >> >> Creating a PEP and creating the library and the PRs can happen in parallel, >> but at least for pip if something deserves a PEP, we're not going to merge a >> PR until that PEP is generally agreed on. However it can be supremely useful >> to have them all going at the same time, because you run into things that >> you didn't really notice until you went to actually implement it. >> >> My other big suggestion would be to e careful about how much you bite off at >> one time. Pip's internal code base is not the greatest, so pulling out >> smaller chunks at a time rather than trying to start right off pulling out a >> big topic is more likely to meet with success. Be cognizant of what the >> dependencies are for the feature you want to implement, because if it has >> dependencies, you'll need to pull them out first before you can pull it out >> OR you'll need to design the API to invert those dependencies so they get >> passed in instead. >> >> I personally would be happy to at a minimum participate on any issue where >> someone was trying to split out some functionality from pip into a re-usable >> library if not follow the develop of that library directly to help guide it >> more closely. My hope for pip is that it ends up being the glue around a >> bunch of these libraries, and that it doesn't implement most of the stuff >> itself anymore. >> -- >> Distutils-SIG mailing list -- distutils-sig@python.org >> <mailto:distutils-sig@python.org> >> To unsubscribe send an email to distutils-sig-le...@python.org >> <mailto:distutils-sig-le...@python.org> >> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ >> <https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/> >> Message archived at >> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/IQVZVVWX2BLEP6D4WQMKNXZHBF2NZINU/ >> >> <https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/IQVZVVWX2BLEP6D4WQMKNXZHBF2NZINU/> > -- > Distutils-SIG mailing list -- distutils-sig@python.org > <mailto:distutils-sig@python.org> > To unsubscribe send an email to distutils-sig-le...@python.org > <mailto:distutils-sig-le...@python.org> > https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ > <https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/> > Message archived at > https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/CIB4OWV2CWR3RVOPTS55WWL4ISFDGTSC/ > > <https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/CIB4OWV2CWR3RVOPTS55WWL4ISFDGTSC/>
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/OFDPKS2EAYQVCKHGSI5BJBKFJT2IMMUT/
