Re: [Distutils] Upcoming changes to PEP 426/440

2013-07-19 Thread Nick Coghlan
On 20 July 2013 03:18, Daniel Holth wrote: > On Fri, Jul 19, 2013 at 11:23 AM, Vinay Sajip wrote: >> >> >>> >>> I would really like to see one more level of nesting: >>> >>> requires : { run : [ ... ], test : [ ... ] } >>> >> >> >> I've already changed distlib's code several times as the spec has

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Nick Coghlan
On 20 July 2013 01:47, PJ Eby wrote: > On Fri, Jul 19, 2013 at 9:10 AM, Nick Coghlan wrote: >> Right, I think the reasonable near term solutions are for pip to either: >> >> 1. generate zc.buildout style wrappers with absolute paths to avoid >> the implied runtime dependency >> 2. interpret use o

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Nick Coghlan
On 20 July 2013 01:20, Brett Cannon wrote: > If the long-term plan is to bless setuptools then go for the bundling, but > if that decision has not been made yet then bundling may be premature if the > bundling of pip with Python moves forward. PEP 426 is currently looking at blessing a subset of

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Nick Coghlan
On 20 July 2013 13:52, Marcus Smith wrote: > >> >> Folks are aware that get-pip.py includes an entire pip installation griped >> inside of it right? >> >> So if we ship get-pip.py we're shipping pip, which we'll then use to >> install pip over the network >> … instead of just shipping pip. > > > y

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Marcus Smith
> What do people think of the idea of my writing something up as an > actual "Python packaging road map" and adding it to the user guide? > (It would probably replace > https://python-packaging-user-guide.readthedocs.org/en/latest/future.html) > > Note that I wouldn't actually *do* this until after

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Nick Coghlan
On 19 July 2013 14:06, Nick Coghlan wrote: > We have a lot of initiatives going every which way at the moment, so I > figured it would be a good idea to get a common perception of what we > consider to be the important near term goals and a realistic timeline > for improving the packaging ecosyste

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Marcus Smith
> > Folks are aware that get-pip.py includes an entire pip installation griped > inside of it right? > > So if we ship get-pip.py we're shipping pip, which we'll then use to > install pip over the network > … instead of just shipping pip. > yes, it has pip in it. I mentioned it as something to im

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Nick Coghlan
On 19 July 2013 14:06, Nick Coghlan wrote: > Independent activities & miscellaneous suggestions > > * maybe suggest "pip install distlib" over pip gaining its own > programmatic API? > * PEP 8 cleanup (including clarification of what constitutes an > internal API) > * improved PyPI upl

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Donald Stufft
On Jul 19, 2013, at 11:27 PM, Nick Coghlan wrote: > On 20 July 2013 02:31, Marcus Smith wrote: >> >>>* decide on a bundling or explicit bootstrapping scheme for pip >>> (this still needs a PEP to help clarify the pros and cons of the >>> various alternatives) >> >> >> if we improve thing

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Nick Coghlan
On 20 July 2013 02:31, Marcus Smith wrote: > >> * decide on a bundling or explicit bootstrapping scheme for pip >> (this still needs a PEP to help clarify the pros and cons of the >> various alternatives) > > > if we improve things enough so that the get-pip.py experience is reliable > and rob

Re: [Distutils] entry points PEP

2013-07-19 Thread PJ Eby
On Fri, Jul 19, 2013 at 2:09 PM, Joe Gordon wrote: > When I try importing pkg_resources in our development environment it is very > slow: Use zc.buildout to install the application you're invoking, and then it won't need to import pkg_resources. (Unless the actual app uses it.) _

[Distutils] wheel without console_scripts in 0.20.0

2013-07-19 Thread Daniel Holth
Hi Paul. https://bitbucket.org/dholth/wheel/commits/db913ecc2c0bb7fec14ba9eae68f99d9a9743bf6 Wheel 0.20.0 won't put entry_points console_scripts and gui_scripts in the wheel itself. The new "python -m wheel install-scripts wheel pip" command for example would install those scripts for wheel and

Re: [Distutils] entry points PEP

2013-07-19 Thread Joe Gordon
On Fri, Jul 19, 2013 at 1:42 PM, PJ Eby wrote: > On Fri, Jul 19, 2013 at 2:09 PM, Joe Gordon wrote: > > When I try importing pkg_resources in our development environment it is > very > > slow: > > Use zc.buildout to install the application you're invoking, and then > it won't need to import pkg_

Re: [Distutils] entry points PEP

2013-07-19 Thread Monty Taylor
Yeah. Not moving to zc.buildout for anything. I believe it will be a better option to just write by-hand scripts that get installed that just do: from nova.rootwrap import cmd return cmd.main(sys.argv) or something. Basically, a tiny boiler-plate script that does the same thing as a console_scri

Re: [Distutils] entry points PEP

2013-07-19 Thread Daniel Holth
On Fri, Jul 19, 2013 at 6:10 PM, Joe Gordon wrote: > > > > On Fri, Jul 19, 2013 at 1:42 PM, PJ Eby wrote: >> >> On Fri, Jul 19, 2013 at 2:09 PM, Joe Gordon wrote: >> > When I try importing pkg_resources in our development environment it is >> > very >> > slow: >> >> Use zc.buildout to install th

Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-19 Thread Steve Dower
> From: Oscar Benjamin > I don't know whether or not you intend to have wrappers also work for > Python 2.7 (in a third-party package perhaps) but there is a slightly > subtle point to watch out for when non-ASCII characters in sys.argv > come into play. > > Python 2.x uses GetCommandLineA and 3.x

Re: [Distutils] entry points PEP

2013-07-19 Thread Joe Gordon
I have gone ahead and gathered some information using our standard development environment, devstack, I ran cProfile on our application, with the contents of it mocked out, http://paste.openstack.org/show/40948/ When I try importing pkg_resources in our development environment it is very slow:

Re: [Distutils] Upcoming changes to PEP 426/440

2013-07-19 Thread Daniel Holth
On Fri, Jul 19, 2013 at 11:23 AM, Vinay Sajip wrote: > > >> >> I would really like to see one more level of nesting: >> >> requires : { run : [ ... ], test : [ ... ] } >> > > > I've already changed distlib's code several times as the spec has evolved, > and would like not to see any more changes

[Distutils] distlib Wheel.install API - proposed changes - distlib users, please comment

2013-07-19 Thread Vinay Sajip
The API for installing wheels in distlib [1] will need changes as a result of recent discussions around the need to have better control over how scripts are generated at installation time. Currently, the API looks like this: def install(self, paths, dry_run=False, executable=None, warner=None,  

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Donald Stufft
On Jul 19, 2013, at 11:20 AM, Brett Cannon wrote: > Which is an argument, in my mind, to vendor setuptools over bundling > (assuming people are using "bundling" as in "install setuptools next to pip > or at least install a .pth file to access the vendored version"). Including > pip with Pytho

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Marcus Smith
> * decide on a bundling or explicit bootstrapping scheme for pip > (this still needs a PEP to help clarify the pros and cons of the > various alternatives) > if we improve things enough so that the get-pip.py experience is reliable and robust (and handles setuptools if not bundled), then migh

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread PJ Eby
On Fri, Jul 19, 2013 at 9:10 AM, Nick Coghlan wrote: > Right, I think the reasonable near term solutions are for pip to either: > > 1. generate zc.buildout style wrappers with absolute paths to avoid > the implied runtime dependency > 2. interpret use of script entry points as an implied dependenc

Re: [Distutils] Wheels and console script entry point wrappers (Was: Replacing pip.exe with a Python script)

2013-07-19 Thread Vinay Sajip
Oscar Benjamin gmail.com> writes: > Python 2.x uses GetCommandLineA and 3.x uses GetCommandLineW. A > wrapper to launch 2.x should use GetCommandLineA and CreateProcessA to > ensure that the 8-bit argument strings are passed through unaltered. > To launch 3.x it should use the W versions. If not

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Vinay Sajip
> >If the long-term plan is to bless setuptools then go for the bundling, but if >that decision has not been made yet then bundling may be premature if the >bundling of pip with Python moves forward. > Well, Nick has said that he thinks that "distlib is the future" (or, I assume, something lik

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Brett Cannon
On Fri, Jul 19, 2013 at 4:31 AM, Nick Coghlan wrote: > On 19 July 2013 18:17, Paul Moore wrote: > > On 19 July 2013 05:06, Nick Coghlan wrote: > >> > >> * (hopefully) add support for indirect imports (see > >> http://mail.python.org/pipermail/import-sig/2013-July/000645.html for > >> the dr

Re: [Distutils] Upcoming changes to PEP 426/440

2013-07-19 Thread Daniel Holth
On Fri, Jul 5, 2013 at 4:25 AM, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: > >> The basic problem with the list form is that allowing two representations >> for the same metadata makes for extra complexity we don't really want. It >> means we have to decide if the decomposed version (3

Re: [Distutils] Upcoming changes to PEP 426/440

2013-07-19 Thread Vinay Sajip
> > I would really like to see one more level of nesting: > > requires : { run : [ ... ], test : [ ... ] } > I've already changed distlib's code several times as the spec has evolved, and would like not to see any more changes so that I can concentrate on some real work ;-) Seriously, wha

Re: [Distutils] API for registering/managing URLs for a package

2013-07-19 Thread M.-A. Lemburg
On 18.07.2013 21:00, Donald Stufft wrote: > Noah, > External urls are still supported (Although discouraged). > > Marc-Andre, > There is documentation in the PEP, however I have another PEP > coming up for a more streamlined upload process that also contains > a much nicer method of

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Brett Cannon
On Thu, Jul 18, 2013 at 8:33 PM, Daniel Holth wrote: > On Thu, Jul 18, 2013 at 8:15 PM, Donald Stufft wrote: > > > > On Jul 18, 2013, at 7:37 PM, Vinay Sajip > wrote: > > > >>> I think the point is that people might be dependent on this > functionality and > >> > >>> changing it out from undern

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Nick Coghlan
On 19 July 2013 19:29, Paul Moore wrote: > On 19 July 2013 05:23, Nick Coghlan wrote: >> >> On 19 July 2013 09:37, Vinay Sajip wrote: >> >> I think the point is that people might be dependent on this >> >> functionality and >> > >> >> changing it out from underneath them could break their world.

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Daniel Holth
On Fri, Jul 19, 2013 at 5:29 AM, Paul Moore wrote: > On 19 July 2013 05:23, Nick Coghlan wrote: >> >> On 19 July 2013 09:37, Vinay Sajip wrote: >> >> I think the point is that people might be dependent on this >> >> functionality and >> > >> >> changing it out from underneath them could break th

Re: [Distutils] entry points PEP

2013-07-19 Thread Daniel Holth
On Fri, Jul 19, 2013 at 5:32 AM, Robert Collins wrote: > On 19 July 2013 21:24, Vinay Sajip wrote: >> Robert Collins robertcollins.net> writes: >> >>> So my question here would be - can we make it faster? We have just >>> been diagnosing a performance problem in nova due to rootwrap being a >>>

Re: [Distutils] entry points PEP

2013-07-19 Thread Paul Moore
On 19 July 2013 09:35, Nick Coghlan wrote: > Not sure, I hadn't even the idea of letting people register arbitrary > "we install this script". Heck, I haven't even worked out what I want > the format to look like :) > That's the big legacy issue. The old distutils script= argument just dumps arb

Re: [Distutils] entry points PEP

2013-07-19 Thread Robert Collins
On 19 July 2013 21:24, Vinay Sajip wrote: > Robert Collins robertcollins.net> writes: > >> So my question here would be - can we make it faster? We have just >> been diagnosing a performance problem in nova due to rootwrap being a >> pkg_resources scripts entry point : just getting to the first l

Re: [Distutils] Q about best practices now (or near future)

2013-07-19 Thread Paul Moore
On 19 July 2013 05:23, Nick Coghlan wrote: > On 19 July 2013 09:37, Vinay Sajip wrote: > >> I think the point is that people might be dependent on this > functionality and > > > >> changing it out from underneath them could break their world. > > > > > > I got the point that Daniel made, and my

Re: [Distutils] entry points PEP

2013-07-19 Thread Vinay Sajip
Robert Collins robertcollins.net> writes: > So my question here would be - can we make it faster? We have just > been diagnosing a performance problem in nova due to rootwrap being a > pkg_resources scripts entry point : just getting to the first line of > main() takes 200ms, and we make dozens o

Re: [Distutils] entry points PEP

2013-07-19 Thread Robert Collins
On 19 July 2013 01:03, Daniel Holth wrote: > Abstract > > This PEP proposes a way to represent the setuptools “entry points” > feature in standard Python metadata. Entry points are a useful > mechanism for advertising or discovering plugins or other exported > functionality without having to depen

Re: [Distutils] entry points PEP

2013-07-19 Thread Nick Coghlan
On 19 July 2013 18:28, Paul Moore wrote: > This is just the metadata. I would assume that the > console-scripts/gui-scripts entry pointdefinitions in setuptools would be > extracted and put into the scripts metadata, and any non-script entry points > would go into exports. The legacy scripts= argu

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Nick Coghlan
On 19 July 2013 18:17, Paul Moore wrote: > On 19 July 2013 05:06, Nick Coghlan wrote: >> >> * (hopefully) add support for indirect imports (see >> http://mail.python.org/pipermail/import-sig/2013-July/000645.html for >> the draft PEP - thanks Eric for taking this from a rough idea in email >>

Re: [Distutils] entry points PEP

2013-07-19 Thread Paul Moore
On 18 July 2013 23:51, Daniel Holth wrote: > On Thu, Jul 18, 2013 at 6:42 PM, Nick Coghlan wrote: > > I actually now plan to make scripts and exports first class citizens in > PEP > > 426, with pydist-scripts.json and pydist-exports.json as extracted > summary > > files (like the existing pydist

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Paul Moore
On 19 July 2013 05:06, Nick Coghlan wrote: > * (hopefully) add support for indirect imports (see > http://mail.python.org/pipermail/import-sig/2013-July/000645.html for > the draft PEP - thanks Eric for taking this from a rough idea in email > to a concrete proposal!) > Looking at the import

Re: [Distutils] Specific packaging goals and a tentative timeline

2013-07-19 Thread Paul Moore
On 19 July 2013 05:06, Nick Coghlan wrote: > * both pip and setuptools available as cross platform wheel files on > PyPI > Just to point out, this is the one that needs a solution to the "script wrappers" issues. At the moment, nominally cross-platform/architecture wheels are actually not so