Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

The short answer is that someone complained about the pip API not doing what they wanted it to do, so we broke the API to make it clear to everyone that the API is not something that should be depended on. 2017-10-20 17:53 GMT-05:00 Richard Jones : > Hiya Paul, > >

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

Hiya Paul, There's a bunch of tooling out there using pip's internals to extending pip's functionality. Could you please provide a some reasoning as to why they're all going to be broken at pip 10, and possibly some guidance on how to get that functionality back? Cheers, Richard On 21

Re: [Distutils] Entry points: specifying and caching

I like the idea of lifecycle hooks but I worry about the malware problem; would there need to be a blacklist / whitelist / disable system? (ignore-scripts=true is now a recommended part of anyone's npm configuration) That is why we have avoided any kind of (package specific) hooks to wheel.

Re: [Distutils] Entry points: specifying and caching

Excerpts from Nathaniel Smith's message of 2017-10-20 13:41:03 -0700: > On Oct 19, 2017 11:10, "Donald Stufft" wrote: > > > EXCEPT, for the fact that with the desire to cache things, it would be > beneficial to “hook” into the lifecycle of a package install. However I > know

Re: [Distutils] Entry points: specifying and caching

On Oct 19, 2017 11:10, "Donald Stufft" wrote: EXCEPT, for the fact that with the desire to cache things, it would be beneficial to “hook” into the lifecycle of a package install. However I know that there are other plugin systems out there that would like to also be able to do

Re: [Distutils] Entry points: specifying and caching

On Friday, October 20, 2017, Doug Hellmann wrote: > Excerpts from Wes Turner's message of 2017-10-20 10:41:02 -0400: > > On Friday, October 20, 2017, Donald Stufft > wrote: > > > > > > > > > > > On Oct 20, 2017, at 9:35 AM, Nick Coghlan

Re: [Distutils] Entry points: specifying and caching

Excerpts from Thomas Kluyver's message of 2017-10-20 19:37:45 +0100: > On Fri, Oct 20, 2017, at 07:24 PM, Doug Hellmann wrote: > > I have been trying to find time to do something like that within > > stevedore for a while to solve some client-side startup performance > > issues with the OpenStack

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

While I understand that pip itself has to be very careful about edge cases and all the pathological things you can do in setup.py, as a higher-level tooling author my priorities are on the happy path UX and speed is a big factor there. So yes, using PackageFinder is potentially inaccurate, but

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

The "installing collected packages" output is not always correct. The "successfully installed [package list]" is always correct (see pypa/pip/issues/4724 for an example). I understand your requirements, but is the practical implication that pip should offer a command that outputs packages that

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

Installing to a temp dir is really not an option for automated tooling (if nothing else, it takes way too long). `pip list --outdated` does already get fairly close to this (and doesn't install anything I suspect you can actually get a lot closer than you think) but it calculates for all

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

A correct dry-run implementation will do about the same amount of work as installing to a temporary directory right now. In the future, that could be optimized, but any patch to the finder doesn't actually detect the requirements correctly (as they're not necessarily known until after the wheels

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 07:31 PM, Marius Gedminas wrote: > Please do not forget about gui_scripts entry points! I haven't forgotten about them in the draft spec: https://github.com/pypa/python-packaging-user-guide/pull/390/files#diff-089b079de062f6fdb759bb719b79e6c8R121

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017 at 08:10:06AM -0400, Donald Stufft wrote: > Packaging tools shouldn’t be expected to know anything about it other > than the console_scripts feature Please do not forget about gui_scripts entry points! Marius Gedminas -- What can I do with Python that I can't do with C#?

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 07:24 PM, Doug Hellmann wrote: > I have been trying to find time to do something like that within > stevedore for a while to solve some client-side startup performance > issues with the OpenStack client. I would be happy to help add it > to entrypoints instead and use it

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

There's no dry-run functionality that I know of so far. However, you could use the following: pip install --prefix=tmpdir This command is actually about the same speed as a proper implementation, because we can't actually know what we're installing until we build the requirements. 2017-10-20

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

> On Oct 20, 2017, at 9:30 AM, Paul Moore wrote: > > On 20 October 2017 at 14:26, Matthew Brett wrote: >> Thanks for the heads-up. >> >> Will y'all be doing a PyPI pre-release so we can test with `pip >> install --pre -U pip`? > > We've not yet

Re: [Distutils] Entry points: specifying and caching

Excerpts from Nick Coghlan's message of 2017-10-20 14:42:09 +1000: > On 20 October 2017 at 02:14, Thomas Kluyver wrote: > > > On Thu, Oct 19, 2017, at 04:10 PM, Donald Stufft wrote: > > > I’m in favor, although one question I guess is whether it should be a a > > > PEP or

Re: [Distutils] Entry points: specifying and caching

Excerpts from Wes Turner's message of 2017-10-20 10:41:02 -0400: > On Friday, October 20, 2017, Donald Stufft wrote: > > > > > > > On Oct 20, 2017, at 9:35 AM, Nick Coghlan > > wrote: > > > > On 20

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

So as someone on the tooling side, is there any kind of install dry-run yet? I've got https://github.com/poise/poise-python/blob/master/lib/poise_python/resources/python_package.rb#L34-L78 which touches a tn of internals. Basically I need a way to know exactly what versions `pip install`

Re: [Distutils] Entry points: specifying and caching

On Wed, 18 Oct 2017 at 17:54 Nick Coghlan wrote: > On 19 October 2017 at 04:18, Alex Grönholm > wrote: > >> Daniel Holth kirjoitti 18.10.2017 klo 21:06: >> >> >>

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

Yeah, +1 from me too. Pip is one project where people will highly likely try out the pre-release versions. xoviat kirjoitti 20.10.2017 klo 20:34: +1 on pre-release wheels. I've seen the process in action with SciPy, and it helped to catch at least one significant bug. 2017-10-20 8:30

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

+1 on pre-release wheels. I've seen the process in action with SciPy, and it helped to catch at least one significant bug. 2017-10-20 8:30 GMT-05:00 Paul Moore : > On 20 October 2017 at 14:26, Matthew Brett > wrote: > > Thanks for the heads-up. > >

Re: [Distutils] Documentation link on PyPI.org

> On 20 Oct 2017, at 17:51, Wes Turner wrote: > > > > On Friday, October 20, 2017, Ronald Oussoren > wrote: > >> On 28 Aug 2017, at 00:12, Brett Cannon >

Re: [Distutils] Documentation link on PyPI.org

On Friday, October 20, 2017, Ronald Oussoren wrote: > > On 28 Aug 2017, at 00:12, Brett Cannon > wrote: > > If you search the archive of this mailing list you will notice I asked > this exact question

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

Perhaps pkg_resources.find_distributions()? http://setuptools.readthedocs.io/en/latest/pkg_resources.html#getting-or-creating-distributions Jannis Gebauer kirjoitti 20.10.2017 klo 16:55: Thanks for the heads-up, Paul. I’m currently using `pip.get_installed_distributions` and as far as I can

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

Thanks for the heads-up, Paul. I’m currently using `pip.get_installed_distributions` and as far as I can see that has moved into `_internal`, too: https://github.com/pypa/pip/blob/master/src/pip/_internal/utils/misc.py#L333

Re: [Distutils] Entry points: specifying and caching

On Friday, October 20, 2017, Donald Stufft wrote: > > > On Oct 20, 2017, at 9:35 AM, Nick Coghlan > wrote: > > On 20 October 2017 at 23:19, Donald Stufft

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

On 20 October 2017 at 14:55, Jannis Gebauer wrote: > Thanks for the heads-up, Paul. > > I’m currently using `pip.get_installed_distributions` and as far as I can > see that has moved into `_internal`, too: >

Re: [Distutils] Documentation link on PyPI.org

> On 28 Aug 2017, at 00:12, Brett Cannon wrote: > > If you search the archive of this mailing list you will notice I asked this > exact question about a month or 2 ago (I think). The answer I got was it is > used on PyPI.org, but I don't know how to set it with setuptools

Re: [Distutils] pythonhosted.org doc upload no longer works

On 20 October 2017 at 22:42, Ronald Oussoren wrote: > > On 7 Oct 2017, at 09:12, Giampaolo Rodola' wrote: > > Any news about this please? This is becoming a problem as my outdated doc > keeps being around and I have no way to remove it. > > > I have

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 9:35 AM, Nick Coghlan wrote: > > On 20 October 2017 at 23:19, Donald Stufft > wrote: > One that I was helping someone debug just the other day is that they’re super > non-debuggable and the behavior when

Re: [Distutils] pythonhosted.org doc upload no longer works

> On 7 Oct 2017, at 09:12, Giampaolo Rodola' wrote: > > Any news about this please? This is becoming a problem as my outdated doc > keeps being around and I have no way to remove it. I have the same problem with my projects: the documentation on pythonhosted.org can no

Re: [Distutils] Entry points: specifying and caching

On 20 October 2017 at 23:19, Donald Stufft wrote: > One that I was helping someone debug just the other day is that they’re > super non-debuggable and the behavior when you have two things providing > the same entry point is basically (If I remember correctly, the >

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

On 20 October 2017 at 14:26, Matthew Brett wrote: > Thanks for the heads-up. > > Will y'all be doing a PyPI pre-release so we can test with `pip > install --pre -U pip`? We've not yet decided on that. Traditionally I don't think we have done so, but I'm inclined to think

Re: [Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

Hi, On Fri, Oct 20, 2017 at 2:22 PM, Paul Moore wrote: > We're in the process of starting to plan for a release of pip (the > long-awaited pip 10). We're likely still a month or two away from a > release, but now is the time for people to start ensuring that > everything

[Distutils] Announcement: Pip 10 is coming, and will move all internal APIs

We're in the process of starting to plan for a release of pip (the long-awaited pip 10). We're likely still a month or two away from a release, but now is the time for people to start ensuring that everything works for them. One key change in the new version will be that all of the internal APIs

Re: [Distutils] Entry points: specifying and caching

On Oct 20, 2017, at 8:41 AM, Thomas Kluyver wrote: > > On Fri, Oct 20, 2017, at 01:36 PM, Donald Stufft wrote: >> Entry points have a lot of problems and I know of multiple systems that have >> either moved away from them, had to hack around how bad they are, have >>

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 01:58 PM, Wes Turner wrote: > What were the issues with setuptools entry points here (in 2014, when > you two were opposed to adding them to sendibly list ipython > extensions)? I'm impressed by your memory! The main issue then was that it implied that extension authors

Re: [Distutils] Entry points: specifying and caching

On Friday, October 20, 2017, Thomas Kluyver wrote: > On Fri, Oct 20, 2017, at 01:36 PM, Donald Stufft wrote: > > Entry points have a lot of problems and I know of multiple systems that > have either moved away from them, had to hack around how bad they are, have > refused

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 01:36 PM, Donald Stufft wrote: > Entry points have a lot of problems and I know of multiple systems > that have either moved away from them, had to hack around how bad they > are, have refused to implement them because of previous pain felt by > them, are looking for ways

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 8:34 AM, Nick Coghlan wrote: > > You're acting like you believe you have veto power over this topic. You don't > - it's not a PyPI related concern, and it doesn't require any changes to pip > or warehouse. > > I'd certainly be *happier* if you were

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 8:23 AM, Nick Coghlan wrote: > > On 20 October 2017 at 22:10, Donald Stufft > wrote: > If I could guess, I’d say it hasn’t changed in years because setuptools has > had bigger things to work on and not

Re: [Distutils] Entry points: specifying and caching

On 20 October 2017 at 22:18, Donald Stufft wrote: > > The “existing installations” is horse shit, because existing > implementations won’t support *any* new feature of anything so it can > literally be used as a justification for doing nothing about anything > except

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 01:18 PM, Donald Stufft wrote: > I guess we shouldn’t have done PEP 517 or PEP 518 because, by your > logic here, since it won’t be supported by existing tooling, there > won’t be any incentive for people to use it ever. I see this as having a similar purpose to those

Re: [Distutils] Entry points: specifying and caching

On 20 October 2017 at 22:10, Donald Stufft wrote: > If I could guess, I’d say it hasn’t changed in years because setuptools > has had bigger things to work on and not enough time to do it in. Then you'd be wrong - it hasn't changed in years because it's a sensible, simple

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 8:06 AM, Nick Coghlan wrote: > > On 20 October 2017 at 21:15, Donald Stufft > wrote: > Tell you what, I’ll drop everything today and write up a PEP that adds > metadata for console scripts to the packaging

Re: [Distutils] Entry points: specifying and caching

On 20 October 2017 at 21:57, Thomas Kluyver wrote: > On Fri, Oct 20, 2017, at 12:50 PM, Donald Stufft wrote: > > * We stifle innovation (hell just including it in setutools at all does > > this, but we can’t unopen that can of worms). > > I don't think that's true to any

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 7:57 AM, Thomas Kluyver wrote: > > On Fri, Oct 20, 2017, at 12:50 PM, Donald Stufft wrote: >> * Since it is a packaging standard, then it is expected that all >> packaging tools will be updated to work with it. > > Where packaging tools need to know

Re: [Distutils] Entry points: specifying and caching

On 20 October 2017 at 21:15, Donald Stufft wrote: > Tell you what, I’ll drop everything today and write up a PEP that adds > metadata for console scripts to the packaging metadata where it belongs, > Donald, you're making the same mistake I did with PEP 426: interoperability

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 12:50 PM, Donald Stufft wrote: > * Since it is a packaging standard, then it is expected that all > packaging tools will be updated to work with it. Where packaging tools need to know about it, they already have to. Where they don't, writing a standard doesn't imply that

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 7:31 AM, Thomas Kluyver wrote: > > On Fri, Oct 20, 2017, at 12:15 PM, Donald Stufft wrote: >> Tell you what, I’ll drop everything today and write up a PEP... > > Donald, why are you so determined that this spec should not be created? Your > time is

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 12:15 PM, Donald Stufft wrote: > Tell you what, I’ll drop everything today and write up a PEP... Donald, why are you so determined that this spec should not be created? Your time is enormously valuable, so why would you drop everything to write a PEP which implies changes

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 1:32 AM, Nick Coghlan wrote: > > 3. Unlike setup.cfg & pyproject.toml, actual humans never touch it - it's > written and read solely by software This is wrong BTW, humans can and do effectively write entry_points.txt, it’s a supported feature of

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 7:02 AM, Nick Coghlan wrote: > > That's the one point where the "de facto standard" status of setuptools is > relevant to the question of whether the entry_points.txt format is a PyPA > interoperability standard: it is, because providing a

Re: [Distutils] Entry points: specifying and caching

On 20 October 2017 at 20:48, Donald Stufft wrote: > > On Oct 20, 2017, at 1:32 AM, Nick Coghlan wrote: > > If we want to enable pytest plugin authors to use other build systems like > flit, then those build systems need a defined interoperability format >

Re: [Distutils] Entry points: specifying and caching

On 20 October 2017 at 16:43, Thomas Kluyver wrote: > I would also be happy to add a section to the document describing the > specific use of entry points for defining scripts to install. > Yeah, it would make sense to include that, as well as reserving the

Re: [Distutils] Entry points: specifying and caching

> On Oct 20, 2017, at 1:32 AM, Nick Coghlan wrote: > > If we want to enable pytest plugin authors to use other build systems like > flit, then those build systems need a defined interoperability format that's > compatible with what pytest is expecting to see (i.e. entry

Re: [Distutils] Entry points: specifying and caching

I would also be happy to add a section to the document describing the specific use of entry points for defining scripts to install. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Entry points: specifying and caching

On Fri, Oct 20, 2017, at 05:42 AM, Nick Coghlan wrote: > I'm wondering if rather than jumping straight to a PEP, it may make > sense to instead initially pursue this idea as a *non-*standard, > implementation dependent thing specific to the "entrypoints" project. > There are a *lot* of challenges