Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Thomas Kluyver
On Fri, May 19, 2017, at 05:17 PM, Paul Moore wrote: > On 19 May 2017 at 16:53, Daniel Holth wrote: > > Congrats on getting 518 in. > > Agreed, by the way. That's a big step! Thanks both. It does feel like an achievement. :-) ___

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Thomas Kluyver
On Fri, May 19, 2017, at 04:31 PM, Paul Moore wrote: > For flit, would having the install tool set PYTHONIOENCODING help? If install tools were meant to set PYTHONIOENCODING, I probably wouldn't do anything else in flit's code. Python should then take care of ensuring that any output is UTF-8

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Thomas Kluyver
On Mon, May 22, 2017, at 11:36 PM, Steve Dower wrote: > IMHO, #2 is definitely the right way to go. Yes, the platform specific > code now has to worry about the encoding, but... the encoding is > platform specific? So... that seems exactly right? :) Maybe I'm still > missing something here, but

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-29 Thread Thomas Kluyver
On Mon, May 29, 2017, at 08:05 AM, Nathaniel Smith wrote: > And what seems like it's really missing is a command to like... > generate an sdist, build it into a wheel, and then leave them both > somewhere twine can see them. Maybe 'pip release .'? FWIW, we've just had a discussion over naming for

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-29 Thread Thomas Kluyver
On Mon, May 29, 2017, at 10:19 AM, Paul Moore wrote: > I think that the key thing here is that as things stand, pip needs a > means to copy an existing "source tree", as efficiently as possible. > For local directories (source checkouts, typically) there's a lot of > clutter that isn't needed to

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Thomas Kluyver
I'm struggling to understand the objections here. As I understand the PEP, the input to building a wheel is a source tree. That may come from an unpacked sdist or a VCS checkout; hopefully those contain basically the same files, give or take some unimportant generated files in an sdist. This seems

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 04:31 PM, Donald Stufft wrote: > The hook is also so a tool like tox or TravisCI or twine can produce a > sdist that can be uploaded to PyPI or similar. This seems like a distinct operation from 'prepare the files needed to build a wheel', although they are related. For

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 03:12 PM, Paul Moore wrote: > I'm struggling to understand why building a sdist in flit should need > a VCS. It bothers me that I'm missing something important about how > backends will work, that explains why (for example) you can't create a > sdist from an export of a VCS

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 03:41 PM, Nick Coghlan wrote: > Instead, the flow that I think makes sense is "VCS -> sdist tree [-> > sdist tree -> sdist tree -> ...] -> wheel", and the above model where > the export filtering is only used when PKG-INFO doesn't exist yet will > give us that. I still

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 05:26 PM, Nick Coghlan wrote: > Could you provide a little more detail as to what you would do > differently in exporting the contents of an sdist that wouldn't apply > to export a build tree? (aside from skipping emitting PKG-INFO) When creating an sdist, I query the VCS

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-01 Thread Thomas Kluyver
On Thu, Jun 1, 2017, at 10:49 PM, Paul Moore wrote: > pip also needs a way to deal with "pip install . In > this case, pip (under its current model) copies that directory to a > working area. In that area, it runs the build command to create a > wheel, and proceeds from there. In principle,

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
As was suggested at some point, I have added a build_sdist hook to my PR, with the following details: - A brief definition of the minimal requirements of an sdist. - I have limited the definition to gzipped tarballs. Zip files also work as sdists, but we're moving towards standardising on

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-02 Thread Thomas Kluyver
On Fri, Jun 2, 2017, at 06:14 PM, Donald Stufft wrote: > The steps here would basically be (for building from something that > isn’t already a .tar.gz or a .whl): That sounds OK to me. I think the only remaining point of contention is whether the build_sdist hook should make an archive or an

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Thomas Kluyver
On Sat, Jun 3, 2017, at 03:14 AM, Nathaniel Smith wrote: > If the pip devs don't trust build systems in general, but (as > suggested by copy_files discussion) are ok with trusting them if they > promise to be super trustworthy, alternate proposal: > - add a 'in_place_build_safe = True' hook, which

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-06-03 Thread Thomas Kluyver
On Sat, Jun 3, 2017, at 10:55 AM, Paul Moore wrote: > This is getting very off-topic, but what if I wanted to patch the > source and then build a sdist to put into my local PyPI index? I > presume the answer is that I either have to checkout the original > sources from VCS or I have to build only

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 09:16 PM, Donald Stufft wrote: > How you build the release-quality sdist isn’t really of concern of PEP > 517 any more than building a release quality wheel is, it’s up to the > build tool to implement that as it makes sense for them. But if we have a hook for building

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 08:20 PM, Donald Stufft wrote: > I am struggling to figure out where there is opposition to simply > exposing something in a standard way, that you were already planning > on implementing anyways. I have issues with it because: 1. Building a *release-quality* sdist is a

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 07:40 PM, Donald Stufft wrote: > Taking pip completely off the table a second, let’s take a look at > tox. Tox’s default mode of operation is to produce a sdist. Now let’s > say I’m writing a project that I want to use PEP 517 and get rid of > setup.py, except now tox is

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 06:03 PM, Donald Stufft wrote: > generating a sdist is an important thing to have be possible, and it > needs to be done in a way that it can be invoked generically. I agree that it needs to be possible to make an sdist, but can we leave the generic interface for it to be

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
I've made an alternative PR against PEP 517 which defines a prepare_build_files hook, defined in terms of copying the files needed to build a package rather than making an sdist. https://github.com/python/peps/pull/277 ___ Distutils-SIG maillist -

[Distutils] Malicious packages on PyPI

2017-06-01 Thread Thomas Kluyver
Are we aware of this? http://evilpackage.fatezero.org/ I recall there were a couple of these before which were taken down, but someone appears to have made a cookiecutter template so you can very easily claim names on PyPI, and anyone who installs that package will submit their information to

Re: [Distutils] Malicious packages on PyPI

2017-06-01 Thread Thomas Kluyver
. On Thu, Jun 1, 2017, at 06:18 PM, Thomas Kluyver wrote: > Are we aware of this? > http://evilpackage.fatezero.org/ > > I recall there were a couple of these before which were taken down, but > someone appears to have made a cookiecutter template so you can very > easily claim names o

Re: [Distutils] Malicious packages on PyPI

2017-06-01 Thread Thomas Kluyver
On Thu, Jun 1, 2017, at 06:32 PM, Matt Joyce wrote: > It's basically a test dummy package that reports users who have ran > that package template. That's what I thought, but all the code to do the upload seems to have been removed before s/he built those packages. Now it's just a harmless warning,

Re: [Distutils] PEP 517: Open questions around artifact export directories

2017-06-13 Thread Thomas Kluyver
On Tue, Jun 13, 2017, at 02:27 AM, Nick Coghlan wrote: > Despite being the one to originally propose standardisation on passing > directory paths around, I'm starting to lean back towards this > approach. > > My rationale for this doesn't really have a lot to do with topics > we've discussed so

[Distutils] Finishing up PEP 517

2017-06-15 Thread Thomas Kluyver
Yes, it's PEP 517 again! Here's the current text: https://www.python.org/dev/peps/pep-0517/ We currently say: > All other hooks [except get_build_requires] are executed in an environment > which contains both the bootstrap requirements specified in the > pyproject.toml hook and those

Re: [Distutils] PEP 517: Open questions around artifact export directories

2017-06-15 Thread Thomas Kluyver
On Thu, Jun 15, 2017, at 03:10 PM, C Anthony Risinger wrote: > I'm not trying to open a bikeshedding opportunity here -- and I tried > to ignore it, honest! -- but why are tarballs preferable to zipfiles > for sdists?> > ... > > Just seems a little odd/arbitrary to me that wheel is zip, python >

Re: [Distutils] Finishing up PEP 517

2017-06-16 Thread Thomas Kluyver
On Fri, Jun 16, 2017, at 10:48 PM, Nathaniel Smith wrote: > The messy complications come from > prepare_wheel_metadata and get_prepare_wheel_input_files, which isn't > surprising, since those are the two hooks where we're squinting into > our crystal ball to try and guess what will be useful for

Re: [Distutils] PEP 517: Python 3 finder?

2017-06-16 Thread Thomas Kluyver
On Fri, Jun 16, 2017, at 10:01 PM, Nathaniel Smith wrote: > On Jun 16, 2017 12:13 PM, "Daniel Holth" wrote: >> Build systems should be able to run under a different version of >> Python than the one that is running 'pip install'. Does PEP 517 have >> anything to say about that?>

Re: [Distutils] PEP 517: Open questions around artifact export directories

2017-06-10 Thread Thomas Kluyver
On Sat, Jun 10, 2017, at 04:32 AM, Nick Coghlan wrote: > 1. Move all artifact naming responsibilities to the frontend. All > backend hooks would just write files in the defined format into the > directory supplied by the frontend (and those directories would have > no defined naming scheme -

Re: [Distutils] Finishing up PEP 517

2017-06-16 Thread Thomas Kluyver
On Fri, Jun 16, 2017, at 08:41 AM, Nick Coghlan wrote: > > I think we should rename get_build_requires to > > get_build_wheel_requires, and add a get_build_sdist_requires. And the > > rule would be: > > > > get_build_sdist_requires: can assume build-system.requires are available > >

Re: [Distutils] PEP 517: Open questions around artifact export directories

2017-06-14 Thread Thomas Kluyver
I have updated the PR as requested: 1. Specify the PAX format, and note that it is not yet the default for the tarfile module in Python. 2. Updated the self-contained backend example with Nick's suggestions. Thomas On Wed, Jun 14, 2017, at 08:53 AM, Paul Moore wrote: > On 14 June 2017 at 02:53,

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 02:24 PM, Nick Coghlan wrote: > Cool, thanks for doing that - I'll let you thrash out the details with > Thomas as the PEP author, and then take a look once you're in > agreement on the amendments you want to make. I've had a quick look over the PR, and my main thought is

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 02:57 PM, Paul Moore wrote: > My feeling is that in the current context we're talking about "minimal > set of files needed to build the wheel". That's not precisely a sdist > (because it ignores additional files that the user might want to > include like README, COPYRIGHT,

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-31 Thread Thomas Kluyver
On Wed, May 31, 2017, at 03:00 PM, Thomas Kluyver wrote: > I would be fine with specifying a hook to copy only the files needed to > build the wheel, but if we do that, let's not call it 'sdist' or > anything that suggests that. Also, if this is all we're after, I'd like to push again f

Re: [Distutils] Provisionally accepting PEP 517's declarative build system interface

2017-05-30 Thread Thomas Kluyver
What about saying that the copying step, if necessary, is part of the build backend's responsibilities? I.e. pip doesn't copy the whole directory to a temporary build location, but the build backend may decide to do that at its discretion when it's asked to build a wheel. pip would continue to

[Distutils] PEP 517 - is it ready?

2017-09-11 Thread Thomas Kluyver
Discussion on PEP 517 seems to have settled down, and I believe that Nick said he was about ready to accept it. Is everyone involved satisfied with the current state? Or is there anything else you think should be considered before accepting it? https://www.python.org/dev/peps/pep-0517/

Re: [Distutils] string types for paths in PEP 517

2017-09-05 Thread Thomas Kluyver
I considered this. It's *potentially* a problem, but I think we should not try to deal with it for now: - Normally, temp files will go in /tmp - so it should be fine to construct paths of entirely ascii characters. - Frontends that want the wheel to end up elsewhere can ask for it in a tmp

Re: [Distutils] Reproducible builds (Sdist)

2017-09-29 Thread Thomas Kluyver
On Fri, Sep 29, 2017, at 07:16 PM, Matthias Bussonnier wrote: > Second; is there a convention to store the SDE value ? I don't seem to > be able to find one. It is nice to have reproducible build; but if > it's a pain for reproducers to find the SDE value that highly decrease > the value of SDE

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-24 Thread Thomas Kluyver
On Thu, Aug 24, 2017, at 03:05 PM, xoviat wrote: > During development, you want to use an editable install, which does > modify the source tree. I think it was before you joined the discussion, but editable installs came up before and have deliberately been left out of PEP 517 because of the

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-24 Thread Thomas Kluyver
On Thu, Aug 24, 2017, at 03:29 PM, xoviat wrote: > I understand that, but what I disagree with is modifying build_wheel > to make up for a lack of consensus on editable installs. The reasoning for the build_directory parameter was not based around editable installs. This is the confusion I was

[Distutils] PEP 517 again

2017-08-24 Thread Thomas Kluyver
Nathaniel seems to be busy with other things at the moment, so I hope he won't mind me passing on this list of things he'd like to resolve with the draft PEP. I'll quote his comments and put my responses inline. > - needs some sort of NotImplemented(Error) handling specified We've gone round

Re: [Distutils] PEP 517 again

2017-08-24 Thread Thomas Kluyver
On Thu, Aug 24, 2017, at 02:21 PM, xoviat wrote: > I mean is this golang or Python? In Python, you raise > notimplementederror. But there's a NotImplemented singleton in Python as well. The argument for using a return value is that the hook code has to deliberately return that, whereas a

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-24 Thread Thomas Kluyver
On Thu, Aug 24, 2017, at 06:20 PM, Daniel Holth wrote: > On Thu, Aug 24, 2017 at 12:34 PM Thomas Kluyver > <tho...@kluyver.me.uk> wrote:>> Is there a reason to ask for an 'unclean' > build, though? There may be >> a performance optimisation from reusing cached data,&

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-24 Thread Thomas Kluyver
On Thu, Aug 24, 2017, at 07:30 PM, Leonardo Rochael Almeida wrote: > It looks like a lot of trouble for a feature that is designed to > solve a problem for a very thin intersection of the Venn diagram of > people who:> > a. wants to quickly iterate while hacking a package > > b. doesn't want

Re: [Distutils] PEP 517 again

2017-08-25 Thread Thomas Kluyver
What if hooks returned instances of the exception class? That way it doesn't bubble up from internal code, it can carry a message, and we can all agree that it's a semantically terrible idea that doesn't fit with any conventions. Even I'm not sure if this is a serious suggestion. ;-) On Fri, Aug

Re: [Distutils] PEP 517 again

2017-08-25 Thread Thomas Kluyver
On Fri, Aug 25, 2017, at 10:02 PM, xoviat wrote: > While we're waiting for others to respond, what are we going to do > with respect to the sys.path issue? I don't think there has been > discussion on that yet. I'm more or less persuaded by Nathaniel's argument that the source directory shouldn't

Re: [Distutils] PEP 517 again

2017-08-25 Thread Thomas Kluyver
On Fri, Aug 25, 2017, at 09:50 PM, xoviat wrote: > Genius! 1% inspiration, 99% frustration :-P ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 again

2017-08-25 Thread Thomas Kluyver
On Fri, Aug 25, 2017, at 09:41 PM, xoviat wrote: > There's no need to return an instance of the exception class. As long > as it is defined in the same name space as the hooks the front end > will be able to access it. No, I meant NotImplementedError. As in: if i_cant_do_it: return

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-25 Thread Thomas Kluyver
On Fri, Aug 25, 2017, at 11:45 AM, Paul Moore wrote: > If it can't build a sdist then there's no opportunity for frontends to > fall back to sdist->wheel anyway, so that's not an issue[1]. > > It's only if sdist->wheel and direct wheel are both possible that > frontends have a choice, and I feel

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-25 Thread Thomas Kluyver
Thanks Paul, On Fri, Aug 25, 2017, at 10:45 AM, Paul Moore wrote: > I'd *also* expect that the wheel built (and hence what gets installed) > would be the same as what you'd get if the user added write > permissions and then ran the command. That's part of what I mean by > direct wheel and

Re: [Distutils] PEP 517 again

2017-08-25 Thread Thomas Kluyver
On Fri, Aug 25, 2017, at 10:26 PM, xoviat wrote: > > I'm more or less persuaded by Nathaniel's argument that the source > > directory shouldn't be on sys.path> > I do too. There should be an option in pyproject.toml to disable this > behavior though so that numpy can build itself. I'm not sure

Re: [Distutils] PEP 517 - a plan of action

2017-08-27 Thread Thomas Kluyver
Thanks Paul for providing us with a way to move forwards. I'm in favour of what you propose. On Sun, Aug 27, 2017, at 11:37 AM, Paul Moore wrote: > 2. Thomas to complete and publish his library of helper functions for > PEP 517 consumers (optional - but I think he said he had such a thing > in

Re: [Distutils] PEP 517 again

2017-08-28 Thread Thomas Kluyver
If pip does uses build_wheel directly, as Paul now prefers, I think we can leave the NotImplemented/Error/None question for a later date. We only want some way to signal "I can't do that" because a frontend might try sdist->wheel with a fallback to making a wheel directly. If no frontend is

Re: [Distutils] PEP 517 again

2017-08-28 Thread Thomas Kluyver
On Mon, Aug 28, 2017, at 08:20 PM, Paul Moore wrote: > Maybe we go fully to Nick's proposal that we don't mandate any sort of > consistency constraints in the PEP. That would mean pip *has* to go > sdist->wheel (because pip does need consistent behaviour), and > xoviat's setuptools backend can

Re: [Distutils] PEP 517 again

2017-08-28 Thread Thomas Kluyver
On Mon, Aug 28, 2017, at 08:50 PM, Paul Moore wrote: > My main motivation for wavering is that I thought agreeing to trust > the backend would simplify many of the decisions, and it's immensely > frustrating to me that we're still debating the same question in the > "return None" thread. The

Re: [Distutils] PEP 517 again

2017-08-28 Thread Thomas Kluyver
On Mon, Aug 28, 2017, at 09:13 PM, Daniel Holth wrote: > Then end the debate by letting the PEP authors decide the return type, > and write a paragraph explaining why the other options were rejected. > It is not going to make a big difference. Will that work now? Are we all so tired of this

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-25 Thread Thomas Kluyver
On Fri, Aug 25, 2017, at 11:56 AM, Thomas Kluyver wrote: > Great, I think we're on the same page. I'm going to make a PR against > the PEP while I still remember what page it is. ;-) https://github.com/python/peps/pull/364 ___ Distutils-SIG ma

Re: [Distutils] Fwd: Re: PEP 517 again

2017-08-25 Thread Thomas Kluyver
On Fri, Aug 25, 2017, at 08:47 AM, Paul Moore wrote: > I've never cared about whether the backend modifies the source tree, > only that the direct-wheel / sdist->wheel equivalence holds. If a user types 'pip install .' in a directory where they don't have write permissions, what should happen?

Re: [Distutils] PEP 517 again

2017-08-25 Thread Thomas Kluyver
Can I gently ask everyone involved to consider whether the notimplemented/error discussion is verging into bikeshedding (http://bikeshed.org/)? The technical arguments I have seen so far are: - The exception can include a message - The return value can't 'bubble up' from the internals of a hook

[Distutils] Entry points: specifying and caching

2017-10-18 Thread Thomas Kluyver
We're increasingly using entry points in Jupyter to help integrate third-party components. This brings up a couple of things that I'd like to do: 1. Specification As far as I know, there's no document describing the details of entry points; it's a de-facto standard established by setuptools. It

Re: [Distutils] Entry points: specifying and caching

2017-10-18 Thread Thomas Kluyver
On Wed, Oct 18, 2017, at 05:59 PM, Paul Moore wrote: > > I've always used the setuptools documentation as a reference. Are you > > suggesting moving that information to a different location to > > allow/encourage other tools to implement it as a standard? > > I've never used entry points myself

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 01:18 PM, Donald Stufft wrote: > I guess we shouldn’t have done PEP 517 or PEP 518 because, by your > logic here, since it won’t be supported by existing tooling, there > won’t be any incentive for people to use it ever. I see this as having a similar purpose to those

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 01:58 PM, Wes Turner wrote: > What were the issues with setuptools entry points here (in 2014, when > you two were opposed to adding them to sendibly list ipython > extensions)? I'm impressed by your memory! The main issue then was that it implied that extension authors

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 01:36 PM, Donald Stufft wrote: > Entry points have a lot of problems and I know of multiple systems > that have either moved away from them, had to hack around how bad they > are, have refused to implement them because of previous pain felt by > them, are looking for ways

Re: [Distutils] Entry points: specifying and caching

2017-10-19 Thread Thomas Kluyver
On Thu, Oct 19, 2017, at 08:01 PM, Donald Stufft wrote: > >> On Oct 19, 2017, at 2:54 PM, Thomas Kluyver >> <tho...@kluyver.me.uk> wrote:>> >> I don't think this needs to be controversial. They are a de-facto >> packaging standard, whether or not that's t

Re: [Distutils] Entry points: specifying and caching

2017-10-19 Thread Thomas Kluyver
On Thu, Oct 19, 2017, at 08:29 PM, Donald Stufft wrote: > Because it is? A generic plugin mechanism is not a packaging feature > any more then a HTTP client is a packaging feature, but setuptools > contains one of those too. Since setuptools was in large part a > packaging library, it will of

Re: [Distutils] Entry points: specifying and caching

2017-10-19 Thread Thomas Kluyver
On Thu, Oct 19, 2017, at 09:04 PM, Donald Stufft wrote: > Like I said, I’m perfectly fine documenting that if you add an > entry_points.txt to the .dist-info directory, that is an INI file that > contains a section named “console_scripts” and define what is valid > inside of the console_scripts

Re: [Distutils] Entry points: specifying and caching

2017-10-19 Thread Thomas Kluyver
On Thu, Oct 19, 2017, at 07:09 PM, Donald Stufft wrote: > So heres a different idea that is a bit more ambitious but that I think > is a better overall idea. Let entrypoints be a setuptools thing, and lets > define some key lifecycle hooks during the installation of a package and > some mechanism

[Distutils] Caching entry points - performance

2017-11-30 Thread Thomas Kluyver
To pick up the caching discussion again, I've started to experiment with a couple of different caching techniques. The headline results: a cold-start scan of entry points goes from about 4.5 seconds with no caching, to 0.45 seconds with caching. There's only a small difference (for me) between

Re: [Distutils] Immutability of Release Metadata in Warehouse

2017-12-19 Thread Thomas Kluyver
On Tue, Dec 19, 2017, at 9:10 PM, Matthias Bussonnier wrote: > One case I could see is the use of the requires_python metadata. It > was not included in the recent release of Django 2.0 (which is py 3 > only) and making a new release will be useless as pip on py2 will > still see Django 2.0.0 as

Re: [Distutils] PEP 517 reference consumer

2017-11-16 Thread Thomas Kluyver
We have now done this, and you can see the new repo here: https://github.com/pypa/pep517 On Thu, Nov 16, 2017, at 01:31 PM, Paul Moore wrote: > I'm happy to (not sure what needs to be done, but I'm sure I can > muddle it through ;-)) > Paul > > On 16 November 2017 at 13:16, Thom

Re: [Distutils] PEP 517 reference consumer

2017-11-16 Thread Thomas Kluyver
On Thu, Nov 16, 2017, at 12:46 PM, Nick Coghlan wrote: > On 16 November 2017 at 21:22, Thomas Kluyver > <tho...@kluyver.me.uk> wrote:>> On Sun, Nov 12, 2017, at 10:51 PM, Nick > Coghlan wrote: >> > +1 from me, too. >> >&

Re: [Distutils] PEP 517 reference consumer

2017-11-16 Thread Thomas Kluyver
On Sun, Nov 12, 2017, at 10:51 PM, Nick Coghlan wrote: > +1 from me, too. Whose further agreement should we get for this? And - if there is agreement - how do we do the transfer? If someone briefly gives me permission to create repos in the PyPA org, I can use Github's transfer feature.

[Distutils] PEP 517 reference consumer

2017-11-12 Thread Thomas Kluyver
I have found a bit more time to work on my PEP 517 consumer API: https://github.com/takluyver/pep517 It provides two main interfaces: - 'pep517.wrappers' is a lower-level interface to call the backend hooks in a subprocess. - 'pep517.envbuild' is a higher-level interface which creates a

Re: [Distutils] PEP 517 reference consumer

2017-11-12 Thread Thomas Kluyver
On Sun, Nov 12, 2017, at 07:21 PM, Paul Moore wrote: > Is it (in your opinion) something that pip might be able to use? The > biggest potential hurdle, I would imagine, is that if pep517.envbuild > uses pip to install dependencies, and pip uses pep517.envbuild, > there's a risk of a recursive loop

Re: [Distutils] RFC: PEP 566 - Metadata for Python Software Packages 1.3

2017-12-08 Thread Thomas Kluyver
Dustin asked me to bring this issue to this thread: Metadata version 1.2 (PEP 345) says that version specifiers within a Requires-Dist field should go in parentheses: "zope.interface (>3.5.0)". The metadata spec on PyPUG repeats this. However, PEP 508 says that the parentheses are not needed,

Re: [Distutils] Entry points: specifying and caching

2017-10-26 Thread Thomas Kluyver
On Sat, Oct 21, 2017, at 07:59 AM, Nick Coghlan wrote: > Yeah, here's the gist of what I had in mind regarding the malware > problem (i.e. aiming to ensure we don't get all of setup.py's problems > back again):> > - a package's own install hooks do *not* get called for that package > - hooks only

Re: [Distutils] [proposal] shared distribution installations

2017-10-30 Thread Thomas Kluyver
On Mon, Oct 30, 2017, at 07:16 PM, RonnyPfannschmidt wrote: > in order to elevate those issues i would like to propose a new > installation layout, > where instead of storing each distribution in every python all > distributions would share a storage, and each individual environment > would only

Re: [Distutils] What's the use case of testpypi?

2017-10-31 Thread Thomas Kluyver
On Tue, Oct 31, 2017, at 12:49 PM, Jeremy Stanley wrote: > Do the two share enough common code for successful uploading to a > devpi instance to be indicative of whether PyPI will accept or > reject on the grounds of, e.g., invalid trove classifiers (this one > in particular has been the most

[Distutils] Re: requirements.txt or not requirements.txt?

2018-05-07 Thread Thomas Kluyver
The recommendation is typically: setup.py for libraries, requirements.txt for applications. But you may also use requirements.txt for specific tasks associated with a library, like building the docs. Donald wrote a blog post which better explains what these two formats

[Distutils] Re: org-mode README file formats

2018-05-08 Thread Thomas Kluyver
On Tue, May 8, 2018, at 5:44 PM, Brett Cannon wrote: > If it is an Emacs thing then I would vote "no" since that's very editor- > specific and I suspect trying to support every plaintext file format > is never-ending. Currently the metadata spec defines the permitted mimetypes. Maybe we should

[Distutils] Re: proposing Python package index upload API spec (potential PEP)

2018-05-08 Thread Thomas Kluyver
I like this practice of writing specifications to better document interfaces that already exist. However, in this case I wonder if it would be better to spend the time defining a new, simpler API instead? I think we're currently in a position where most tools could use a new upload API

[Distutils] Re: org-mode README file formats

2018-05-09 Thread Thomas Kluyver
On Wed, May 9, 2018, at 7:22 PM, Diane Trout wrote: > Does the warehouse actually read the readme file directly? or does it > preferentially look at long_description metadata? I think it only looks at the long_description metadata; as far as I know README is just another file in the archive. If

[Distutils] Re: sudo pip install: install pip files into /usr/local on Linux?

2018-05-25 Thread Thomas Kluyver
On Fri, May 25, 2018, at 5:11 PM, Victor Stinner wrote: > As an user, I want to use "sudo pip install" because packages > installed in /usr (or /usr/local) are accessible without having to > touch PYTHONPATH: the install directory is part of the default > sys.path. This is also true for "pip

[Distutils] Re: sudo pip install: install pip files into /usr/local on Linux?

2018-05-25 Thread Thomas Kluyver
On Fri, May 25, 2018, at 6:58 PM, Wes Turner wrote: > ~/.local/bin is user-writeable. If ~/.local was on PATH or by default, > it could potentially preempt/modify the behavior of system libraries > and binaries; which is a security risk. I've heard this argument before, and it doesn't stand up,

[Distutils] Re: PyPI not showing latest version?

2018-06-13 Thread Thomas Kluyver
Since the old PyPI was shut down, I have noticed that it takes a few minutes for new package updates to show up. I assume that the new site is more heavily cached. On Wed, Jun 13, 2018, at 4:01 PM, Vinay Sajip via Distutils-SIG wrote: > I just uploaded python-gnupg 0.4.3 to PyPI using Twine.

[Distutils] Re: sudo pip install: install pip files into /usr/local on Linux?

2018-05-31 Thread Thomas Kluyver
On Thu, May 31, 2018, at 4:39 PM, Matthias Klose wrote: > It's on the path by default in Debian and Ubuntu, only for new users > (~/.profile). Thanks, that's good to hear. I realise it's been a couple of years since I set up a fresh Linux installation. Upgrades are working too well! --

[Distutils]Re: Possible change to PEP 517: look up the backend as $BACKEND.__build_backend__?

2018-06-24 Thread Thomas Kluyver
On Sun, Jun 24, 2018, at 7:19 AM, Nathaniel Smith wrote: > What do you think? (Thomas, I'd love your thoughts in particular :-).) I agree that it looks nicer, but I'm not sure that it's worth the added complexity: is 'flit' equivalent to 'flit.__build_api__' (i.e. from flit import

[Distutils]Re: Possible change to PEP 517: look up the backend as $BACKEND.__build_backend__?

2018-06-24 Thread Thomas Kluyver
On Sun, Jun 24, 2018, at 9:30 AM, Nathaniel Smith wrote: > (What does 'flit init' do if someone already has a pyproject.toml, by the > way?) At present, it prompts you to overwrite it or cancel. I guess that will need to change at some point as more tools use pyproject.toml. -- Distutils-SIG

[Distutils] Re: (Final) PyPI/Warehouse weekly report: legacy is shut down

2018-05-02 Thread Thomas Kluyver
Thanks Sumana & the team for all your work, and for these updates! On Wed, May 2, 2018, at 2:09 AM, Sumana Harihareswara wrote: > As I announced yesterday[1], here and on the pypi-announce[2] and > general Python announcement[3] lists, we have shut down > legacy.pypi.org, on schedule. (See the

[Distutils] Re: This list will soon be migrating to Mailman 3

2018-04-30 Thread Thomas Kluyver
On Mon, Apr 30, 2018, at 7:37 PM, Wes Turner wrote: > The signature footer from the message from you that ends with "I think > it's fixed now." appears to collapse in the Gmail interface,> but no longer > does with the new, helpful footer? I think the collapsing is a heuristic based on text seen

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 12:15 PM, Donald Stufft wrote: > Tell you what, I’ll drop everything today and write up a PEP... Donald, why are you so determined that this spec should not be created? Your time is enormously valuable, so why would you drop everything to write a PEP which implies changes

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 12:50 PM, Donald Stufft wrote: > * Since it is a packaging standard, then it is expected that all > packaging tools will be updated to work with it. Where packaging tools need to know about it, they already have to. Where they don't, writing a standard doesn't imply that

Re: [Distutils] Entry points: specifying and caching

2017-10-19 Thread Thomas Kluyver
On Thu, Oct 19, 2017, at 04:10 PM, Donald Stufft wrote: > I’m in favor, although one question I guess is whether it should be a a > PEP or an ad hoc spec. Given (2) it should *probably* be a a PEP (since > without (2), its just another file in the .dist-info directory and that > doesn’t actually

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 05:42 AM, Nick Coghlan wrote: > I'm wondering if rather than jumping straight to a PEP, it may make > sense to instead initially pursue this idea as a *non-*standard, > implementation dependent thing specific to the "entrypoints" project. > There are a *lot* of challenges

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
I would also be happy to add a section to the document describing the specific use of entry points for defining scripts to install. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 07:24 PM, Doug Hellmann wrote: > I have been trying to find time to do something like that within > stevedore for a while to solve some client-side startup performance > issues with the OpenStack client. I would be happy to help add it > to entrypoints instead and use it

Re: [Distutils] Entry points: specifying and caching

2017-10-20 Thread Thomas Kluyver
On Fri, Oct 20, 2017, at 07:31 PM, Marius Gedminas wrote: > Please do not forget about gui_scripts entry points! I haven't forgotten about them in the draft spec: https://github.com/pypa/python-packaging-user-guide/pull/390/files#diff-089b079de062f6fdb759bb719b79e6c8R121

Re: [Distutils] Entry points: specifying and caching

2017-10-26 Thread Thomas Kluyver
On Thu, Oct 26, 2017, at 03:57 PM, Daniel Holth wrote: > Would something as simple as a file per sys.path with the 'last > modified by installer' date be helpful? You could check those to > determine whether your cache was out of date. I wonder if we could use the directory mtime for this? It's

[Distutils] PEP 566 - metadata 1.3 changes

2018-01-10 Thread Thomas Kluyver
I hope everyone has had a good break. :-) I'd like to see PEP 566 move forwards. From the last thread, I think people were mostly happy with it as stands, but there were some proposals to introduce further metadata changes. I'd suggest that we finalise the PEP as it currently stands, and save

<    1   2   3   >