Re: [Distutils] Adding entry points into Distutils ?
Paul Moore wrote: 2009/5/9 Chris Withers ch...@simplistix.co.uk: +1. flameAnyone who's arguing against this is either not deploying stuff in a repeatable fashion, and so isn't serious in my books, or is so serious that they're cutting vm images to roll out and so dump everything for the app in site packages/flame Hmm. I'll accept the flame to the extent that I don't deploy stuff to a wide enough audience to be qualified to comment on that side of things. But: 1. Can you clarify what this is? Given that I'm writing this offline on a train, the answer is no, I'm afraid ;-) Part of the issue I see is that there's never a clear enough statement of a proposal for a non-expert in the field to follow. Which proposal do you think isn't clear enough? 2. Distutils is for distributing python modules/packages, so *application* deployment is out of scope. Python applications rely on packages and modules. Application in the scope I'm using it might mean something like YouTube. Script support is for small-scale stuff (in my view). Wrong. I use it for creating start/stop scripts for services along with scripts to stick in crontabs. I don't need to bother with that for small-scales stuff. 3. Accepting that I don't know what you mean by this, can I point out that as a user, I personally have problems with a significant proportion of scripts distributed with packages - so are you saying that those packages aren't serious, Probably, or certainly that their authors don't really care about the scripts they generate... or that there is no way of doing what they (and I) want at present? I'm afraid I don't remember/understand what you want... I'll see if I can find some in due course. (Basically, the types of things I see are scripts distributed on Windows with no filetype extension, .bat wrappers for command line scripts which don't work right when called from another bat file, setuptools-built exe wrappers which result in version-specific binaries for pure-python code, to give some examples). The last is the right thing. These scripts work cross-platform and are tied to the correct version of python for the setup as originally intended, ie: the version the package was installed with. cheers, Chris ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Eric Smith wrote: Paul Moore wrote: 2009/5/7 Tres Seaver tsea...@palladion.com: Eric Smith wrote: Yes. It creates a .exe wrapper [1]. By using entry points, I don't need to care what the target system is. Also, /usr/bin/env might invoke the wrong python. Exactly: using entry points for console scripts guarantees that the python into which the corresponding distribution is installed is the one used to run the script, which is *highly* desirable. Otherwise, you end up with the just install everything in the system Python's site-packages mess. ... and somewhere around here we end up with what I described as an over-engineered solution. Or, what I call a minimal set of required functionality. +1. flameAnyone who's arguing against this is either not deploying stuff in a repeatable fashion, and so isn't serious in my books, or is so serious that they're cutting vm images to roll out and so dump everything for the app in site packages/flame Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Noah Gift wrote: I don't understand what you mean here, either. In film enviroments the whole way we work is by toggling between different enviroments. A developer, artists, etc, will need to work on a specific shot in a movie, and they also need to toggle between films, etc. Have you guys looked at buildout? I know the docs suck, but it strikes me that it might solve a lot of the problems you're complaining about... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Noah Gift wrote: 3. There doesn't seem to be a clean way to inject user specific environment details to the console script. os.environ? Either than, or I'm not following you... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
2009/5/9 Chris Withers ch...@simplistix.co.uk: +1. flameAnyone who's arguing against this is either not deploying stuff in a repeatable fashion, and so isn't serious in my books, or is so serious that they're cutting vm images to roll out and so dump everything for the app in site packages/flame Hmm. I'll accept the flame to the extent that I don't deploy stuff to a wide enough audience to be qualified to comment on that side of things. But: 1. Can you clarify what this is? Part of the issue I see is that there's never a clear enough statement of a proposal for a non-expert in the field to follow. 2. Distutils is for distributing python modules/packages, so *application* deployment is out of scope. Script support is for small-scale stuff (in my view). The fact that it gets used for more doesn't mean it's appropriate... 3. Accepting that I don't know what you mean by this, can I point out that as a user, I personally have problems with a significant proportion of scripts distributed with packages - so are you saying that those packages aren't serious, or that there is no way of doing what they (and I) want at present? Concrete examples of specific packages would probably help. Right this moment, I can't personally provide any because I recently had to rebuild my PC and haven't yet reinstalled the vast number of tried-once-but-never-used-again packages I used to have available... I'll see if I can find some in due course. (Basically, the types of things I see are scripts distributed on Windows with no filetype extension, .bat wrappers for command line scripts which don't work right when called from another bat file, setuptools-built exe wrappers which result in version-specific binaries for pure-python code, to give some examples). Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Withers wrote: Noah Gift wrote: I don't understand what you mean here, either. In film enviroments the whole way we work is by toggling between different enviroments. A developer, artists, etc, will need to work on a specific shot in a movie, and they also need to toggle between films, etc. Have you guys looked at buildout? I know the docs suck, but it strikes me that it might solve a lot of the problems you're complaining about... The docs have gotten a lot better lately, thanks largely to the efforts of Baiju M, who has done a lot of the gardening to get the project site up and useful: http://www.buildout.org/ Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKBcT5+gerLs4ltQ4RAljeAJ9dnfZRiKLtt+9YgRHdpA0V0FxHlwCfWO7V 5EQmUxX+t2DBGnG7aguXvnQ= =dib9 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
There are *many* benefits of adding entry points into Distutils. In
fact, adding a plugin system in there,
will help make the commands more extendable and therefore will help us
in the long term to remove things
out of Distutils.
So any strong opinion against them ?
If not, I'll add a section in PEP 376 - and the first use case I can
see is having a pluggable way to extend
the list of files contained in .egg-info.
The work to be done would be to extract entry points from
pkg_resources (the discovery work that consists of looking
for entry_points.txt files will be covered by the APIs already
described in that PEP) This is not hard.
Yes, I'd like to see entry_points added to Distutils!
I'll also mention my most common use-case for using entry_points is
installing
console_scripts using zc.recipe.egg. This use-case only needs
discovery of
entry_points. Once a distribution's entry_points are made available,
it's
entirely up to the application installer (eg. zc.recipe.egg) as to
what is
done with the entry_point information.
For example, let's say that I have a Python project which I want to
use
zest.releaser for. The entry_points for this package is:
entry_points={
'console_scripts': [
'release = zest.releaser.release:main',
'prerelease = zest.releaser.prerelease:main',
'postrelease = zest.releaser.postrelease:main',
'fullrelease = zest.releaser.fullrelease:main',
'longtest = zest.releaser.longtest:main',
'lasttagdiff = zest.releaser.lasttagdiff:main'],
}
Then I can use buildout to install this package into a project's
deployment
with:
[buildout]
parts = releaser
[releaser]
recipe = zc.recipe.egg
eggs = zest.releaser
Where the zc.recipe.egg recipe will turn all 'console_scripts'
entry_points
into scripts in the ./bin directory. Which I find pretty nice :)
Anyways, I'm mentioning this use-case just to re-iterate that there is
a
difference between asking a distribution what entry_points it provides
(discovery) and deciding what configuration or actions to take based
on that
information (activation or script generation). Also, as for discovery,
this
use-case only requires discovery on a per-distribution basis, it
doesn't
need a 'iterate over all distribution's entry_points installed in some
location' feature.
Also, using console_scripts for entry_points means that there is a
second
way of specifying scripts, since Distutils already has the 'scripts'
metadata field. I would say that the 'scripts' field should then be
deprecated, but perhaps there are reason's beyond breaking backwards
compatability for keeping that field around?
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Thu, May 7, 2009 at 7:18 AM, P.J. Eby p...@telecommunity.com wrote: At 08:28 PM 5/6/2009 +0200, Hanno Schlichting wrote: Doug Hellmann wrote: On May 6, 2009, at 1:46 PM, P.J. Eby wrote: At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote: On May 5, 2009, at 10:50 PM, P.J. Eby wrote: At 12:03 PM 5/6/2009 +1000, Ben Finney wrote: I don't see any advantage, in the context of this discussion, to having an additional, incompatible naming for full-path-to-a-class. Setuptools doesn't limit an entry point to being a class, function, or other top-level name within a module. It can be a method of a class, or an attribute of an attribute. The ':' removes any ambiguity as to which part of the name is the module, and which parts are attributes within that module. Is that level of complexity useful in practice? I can understand how it came to be implemented, but is it actually used by any projects? I use it; I'm not sure who else does. The particular use case I have (and that's most likely to be shared) is that the calling app or framework wants a callable or function, but the providing app or library implements that callable as a classmethod for convenience. That's pretty much what I expected. It feels a little messy to have plugins exposing internals like that but not so much so that I propose we don't allow it. The : syntax seems like the right way to go. I'd be tempted to call this an edge-case. You should be able to expose the internal detail you'd need via a module scope alias for the particular case. That seems easier than providing a whole new notion. Ah, now you reminded me of a forgotten detail: the specific use case was that I had a library with a Command class that was widely used as a base class; simply adding a classmethod to that base class allowed all the existing classes to be used as plugins without any change to their modules' source code: the entry points just always reference the classmethod. [slightly off topic] It seems like registration is the canonical use case for classmethods in Python no? Tarek has a good example in Expert Python Programming of this in his observer example. [/end] ___ Distutils-SIG maillist - distutils-...@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Cheers, Noah ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 6, 2009, at 9:38 PM, Wheat wrote: I'll also mention my most common use-case for using entry_points is installing console_scripts using zc.recipe.egg. I'm curious about that because I've never understood the benefit of using entry points for console scripts. Why not just list the script as a part of the package and have it installed normally? Also, using console_scripts for entry_points means that there is a second way of specifying scripts, since Distutils already has the 'scripts' metadata field. I would say that the 'scripts' field should then be deprecated, but perhaps there are reason's beyond breaking backwards compatability for keeping that field around? I would argue the other way. Why force authors of console scripts to deal with entry points instead of just installing the script as-is? Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
2009/5/7 Doug Hellmann doug.hellm...@gmail.com: I would argue the other way. Why force authors of console scripts to deal with entry points instead of just installing the script as-is? Please explain as-is with reference to ensuring that the script works cross-platform. I think the benefit of entry points for scripts is that it generates appropriate wrappers to allow use on all platforms. Having said that, I find setuptools entry points to be over-engineered, and the Windows wrappers (in particular, the fact that they are not version-independent) to be somewhat clumsy. But as a concept, I like the idea of having a way of specifying that a script is intended as an executable, and having distutils do the job of generating whatever platform cruft is required [1] to make that work. Of course, for even remotely modern Python versions, I'd argue strongly that packages shouldn't be including console scripts, but should rather be supplying modules that can be run as scripts, via the -m argument to python. Users can then build aliases, shell scripts, or whatever is appropriate based on that. Paul. [1] And note especially that .bat files are *not* suitable wrappers on Windows, in spite of the fact that they are commonly used. Their biggest disadvantage is that they don't nest. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 7, 2009, at 8:54 AM, Paul Moore wrote: 2009/5/7 Doug Hellmann doug.hellm...@gmail.com: I would argue the other way. Why force authors of console scripts to deal with entry points instead of just installing the script as-is? Please explain as-is with reference to ensuring that the script works cross-platform. I think the benefit of entry points for scripts is that it generates appropriate wrappers to allow use on all platforms. I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/ bin or somewhere in the path on Windows and call it as hello.py. Does setuptools give me something extra for Windows? I'm not a regular Windows user, so it's likely that there are features I don't know about. Having said that, I find setuptools entry points to be over-engineered, and the Windows wrappers (in particular, the fact that they are not version-independent) to be somewhat clumsy. But as a concept, I like the idea of having a way of specifying that a script is intended as an executable, and having distutils do the job of generating whatever platform cruft is required [1] to make that work. That's the part I'm not clear about. What cruft is needed anywhere? Of course, for even remotely modern Python versions, I'd argue strongly that packages shouldn't be including console scripts, but should rather be supplying modules that can be run as scripts, via the -m argument to python. Users can then build aliases, shell scripts, or whatever is appropriate based on that. Paul. [1] And note especially that .bat files are *not* suitable wrappers on Windows, in spite of the fact that they are commonly used. Their biggest disadvantage is that they don't nest. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
2009/5/7 Doug Hellmann doug.hellm...@gmail.com: I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/bin or somewhere in the path on Windows and call it as hello.py. That works but a lot of Unix users have in the past objected to having '.py' in the name. People then start trying to cater for them by leaving off the '.py', which means it doesn't work on Windows, etc etc. As a Windows user, I'm all in favour of this. But can you persuade the Unix users to agree with me, please? :-) Does setuptools give me something extra for Windows? I'm not a regular Windows user, so it's likely that there are features I don't know about. I don't think so, as such. It gives Unix and Windows users who care (== not me, and clearly not you, either) the ability to call the command hello rather than hello.py. Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Thu, May 07, 2009 at 03:03:29PM +0100, Paul Moore wrote: 2009/5/7 Doug Hellmann doug.hellm...@gmail.com: Why make me define an entry point for that? I can just put it in /usr/bin or somewhere in the path on Windows and call it as hello.py. That works but a lot of Unix users have in the past objected to having '.py' in the name. People then start trying to cater for them by leaving off the '.py', which means it doesn't work on Windows, etc etc. Why can't the install_scripts command from distutils install with a .py extension on windonws and without on UNIX? (Or rather: what would be wrong with such an approach?) Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Eric Smith wrote: Doug Hellmann wrote: On May 7, 2009, at 8:54 AM, Paul Moore wrote: 2009/5/7 Doug Hellmann doug.hellm...@gmail.com: I would argue the other way. Why force authors of console scripts to deal with entry points instead of just installing the script as-is? Please explain as-is with reference to ensuring that the script works cross-platform. I think the benefit of entry points for scripts is that it generates appropriate wrappers to allow use on all platforms. I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/bin or somewhere in the path on Windows and call it as hello.py. Does setuptools give me something extra for Windows? I'm not a regular Windows user, so it's likely that there are features I don't know about. Yes. It creates a .exe wrapper [1]. By using entry points, I don't need to care what the target system is. Also, /usr/bin/env might invoke the wrong python. Exactly: using entry points for console scripts guarantees that the python into which the corresponding distribution is installed is the one used to run the script, which is *highly* desirable. Otherwise, you end up with the just install everything in the system Python's site-packages mess. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKAu4++gerLs4ltQ4RAjW4AJ0cM5oqDL35qoBq1nnl4YD1RmX1NACghbqM Zom9yb5WCzaXFrMz7V75zQU= =CZLG -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 7, 2009, at 10:03 AM, Paul Moore wrote: 2009/5/7 Doug Hellmann doug.hellm...@gmail.com: Does setuptools give me something extra for Windows? I'm not a regular Windows user, so it's likely that there are features I don't know about. I don't think so, as such. It gives Unix and Windows users who care (== not me, and clearly not you, either) the ability to call the command hello rather than hello.py. Hmm, yeah, I see that as useful. I don't work with Windows a lot myself, but I don't want to explicitly drop support there, either. Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
2009/5/7 Tres Seaver tsea...@palladion.com: Eric Smith wrote: Doug Hellmann wrote: On May 7, 2009, at 8:54 AM, Paul Moore wrote: 2009/5/7 Doug Hellmann doug.hellm...@gmail.com: I would argue the other way. Why force authors of console scripts to deal with entry points instead of just installing the script as-is? Please explain as-is with reference to ensuring that the script works cross-platform. I think the benefit of entry points for scripts is that it generates appropriate wrappers to allow use on all platforms. I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/bin or somewhere in the path on Windows and call it as hello.py. Does setuptools give me something extra for Windows? I'm not a regular Windows user, so it's likely that there are features I don't know about. Yes. It creates a .exe wrapper [1]. By using entry points, I don't need to care what the target system is. Also, /usr/bin/env might invoke the wrong python. Exactly: using entry points for console scripts guarantees that the python into which the corresponding distribution is installed is the one used to run the script, which is *highly* desirable. Otherwise, you end up with the just install everything in the system Python's site-packages mess. ... and somewhere around here we end up with what I described as an over-engineered solution. By trying to satisfy everyone's requirements, you ultimately satisfy no-one's. Sigh. I keep meaning to avoid getting sucked back into this tar-pit, and I keep failing :-( Just put me down as a hearty +1 for Doug's just deploy a script called whatever.py approach for standalone stuff, and using python -m for scripts distributed as part of larger distributions. Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 7, 2009, at 10:20 AM, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Eric Smith wrote: Doug Hellmann wrote: On May 7, 2009, at 8:54 AM, Paul Moore wrote: 2009/5/7 Doug Hellmann doug.hellm...@gmail.com: I would argue the other way. Why force authors of console scripts to deal with entry points instead of just installing the script as-is? Please explain as-is with reference to ensuring that the script works cross-platform. I think the benefit of entry points for scripts is that it generates appropriate wrappers to allow use on all platforms. I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/bin or somewhere in the path on Windows and call it as hello.py. Does setuptools give me something extra for Windows? I'm not a regular Windows user, so it's likely that there are features I don't know about. Yes. It creates a .exe wrapper [1]. By using entry points, I don't need to care what the target system is. Also, /usr/bin/env might invoke the wrong python. Exactly: using entry points for console scripts guarantees that the python into which the corresponding distribution is installed is the one used to run the script, which is *highly* desirable. Otherwise, you end up with the just install everything in the system Python's site-packages mess. pip installs my scripts into a virtualenv without any issue and without using entry points, AFAICT. I guess if we move to requiring entry points and disallowing simple script distribution I'll need to find another way to package virtualenvwrapper. Since it's a bash script, it doesn't have entry points. I've been using setuptools to package it so it can be installed via easy_install, since it is a Python development tool. Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/ bin or somewhere in the path on Windows and call it as hello.py. With an entry point, hello.py could simply be: def main(): print 'hello!' But I agree, the Distutils 'scripts' is conceptually simpler. Especially since it let's you run a script 'as-is' without first running an install tool to generate a script wrapper. It's only once a script has dependencies on other Python libraries where relying on those libraries being available for import may not work. In this case, the 'scripts' field forces the user to install those libraries in a global location (site-packages), this isn't always desirable though! As for including BASH scripts in the 'scripts' field ... I dunno. Clearly entry_points are meant to only annotate Python code! Is including BASH scripts in the 'scripts' field an abuse of Distutils or is it acceptable usage? The Distutils docs aren't explicit on this, so I'm inclined to say it's acceptable, so this is at least one good reason for keeping the 'scripts' field. But then having both 'scripts' and 'entry_points/console_scripts' is less than perfect since it introduces (mostly) unneccessary TIMTOWTDI. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Doug Hellmann doug.hellm...@gmail.com writes: I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/ bin or somewhere in the path on Windows and call it as hello.py. To address the issue raised elsewhere in this thread: Why should the name of a *command* include the suffix ‘.py’? That just begs for the situation down the line where one of these commands is being used widely, by the name ‘hello.py’, and then someone wants to provide an alternative implementation in another language. A command implemented in Ruby named ‘hello.py’ (for backward compatibility) is even more annoying that one implemented in Python. This situation is entirely predictable, so naming it *without* a suffix is the right way to go. It also makes the command less annoying to type. The simple rule of thumb: If it's primarily a module to be imported, name it ‘foo.py’, don't mark it executable, and don't use the shebang line. If it's primarily a command to be run as the main program, name it ‘foo’ with no ‘.py’ suffix, mark it executable, and use the shebang line. -- \ “I have had a perfectly wonderful evening, but this wasn't it.” | `\ —Groucho Marx | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Paul Moore wrote: 2009/5/7 Tres Seaver tsea...@palladion.com: Eric Smith wrote: Yes. It creates a .exe wrapper [1]. By using entry points, I don't need to care what the target system is. Also, /usr/bin/env might invoke the wrong python. Exactly: using entry points for console scripts guarantees that the python into which the corresponding distribution is installed is the one used to run the script, which is *highly* desirable. Otherwise, you end up with the just install everything in the system Python's site-packages mess. ... and somewhere around here we end up with what I described as an over-engineered solution. Or, what I call a minimal set of required functionality. By trying to satisfy everyone's requirements, you ultimately satisfy no-one's. Not sure I agree, but in any event that's certainly not a reason to try and make a generally useful solution. I think entry points and scripts are two issues that shouldn't be conflated. Entry points are generally useful (to me and others), and having a way of the installer knowing about scripts and generating correct wrappers for them (in a cross-platform way) is also generally useful (to me and others). That scripts currently use entry points is a design detail that perhaps doesn't need to be exposed. Eric. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Fri, May 8, 2009 at 10:04 AM, Eric Smith e...@trueblade.com wrote: Paul Moore wrote: 2009/5/7 Tres Seaver tsea...@palladion.com: Eric Smith wrote: Yes. It creates a .exe wrapper [1]. By using entry points, I don't need to care what the target system is. Also, /usr/bin/env might invoke the wrong python. Exactly: using entry points for console scripts guarantees that the python into which the corresponding distribution is installed is the one used to run the script, which is *highly* desirable. Otherwise, you end up with the just install everything in the system Python's site-packages mess. ... and somewhere around here we end up with what I described as an over-engineered solution. Or, what I call a minimal set of required functionality. By trying to satisfy everyone's requirements, you ultimately satisfy no-one's. Not sure I agree, but in any event that's certainly not a reason to try and make a generally useful solution. I think entry points and scripts are two issues that shouldn't be conflated. Entry points are generally useful (to me and others), and having a way of the installer knowing about scripts and generating correct wrappers for them (in a cross-platform way) is also generally useful (to me and others). Two current problems I have with console scripts with setuptools are: 1. Different versions of Python conflict with previous versions of console scripts. Take paste for example. 2. The entry point mechanism IIRC recursively scans the site-packages directory and loads up the system path with eggs. This is too expensive of an operation for the current environment I work in. 3. There doesn't seem to be a clean way to inject user specific environment details to the console script. I often need the ability to alter the sys.path in a user specific way for the entry point without needing to mess up the global sys.path permanently. scripts currently use entry points is a design detail that perhaps doesn't need to be exposed. Eric. ___ Distutils-SIG maillist - distutils-...@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Cheers, Noah ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 7, 2009, at 5:53 PM, Ben Finney wrote: Doug Hellmann doug.hellm...@gmail.com writes: I write a python script call hello.py like this: #!/usr/bin/env python def main(): print 'hello!' if __name__ == '__main__': main() Why make me define an entry point for that? I can just put it in /usr/ bin or somewhere in the path on Windows and call it as hello.py. To address the issue raised elsewhere in this thread: Why should the name of a *command* include the suffix ‘.py’? That just begs for the situation down the line where one of these commands is being used widely, by the name ‘hello.py’, and then someone wants to provide an alternative implementation in another language. I personally don't care if it includes the .py suffix or not, most of the time. I tend not to include it in my own code, and don't get bent out of shape if someone else's tool does use it. Of course, as I've been reminded in this thread not including the .py (and not using entry points) means my scripts don't work as-is when installed on Windows. Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 10:21 AM 5/8/2009 +1200, Noah Gift wrote: 1. Different versions of Python conflict with previous versions of console scripts. Take paste for example. I don't understand what you mean. 2. The entry point mechanism IIRC recursively scans the site-packages directory and loads up the system path with eggs. This is too expensive of an operation for the current environment I work in. The scan is not recursive; only files that are actually *on* sys.path are scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info in a directory that is directly on sys.path. Now, it's possible that some application you are using does such a scan explicitly; I'm just noting that merely querying or loading entry points doesn't cause any recursive scans, and it most definitely does not add anything new to sys.path, unless the entry point to be loaded has declared an additional dependency that's *not* on sys.path yet. 3. There doesn't seem to be a clean way to inject user specific environment details to the console script. I often need the ability to alter the sys.path in a user specific way for the entry point without needing to mess up the global sys.path permanently. I don't understand what you mean here, either. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
resending, as I accidently only sent to PJE
On Fri, May 8, 2009 at 11:48 AM, Noah Gift noah.g...@gmail.com wrote:
On Fri, May 8, 2009 at 11:24 AM, P.J. Eby p...@telecommunity.com wrote:
At 10:21 AM 5/8/2009 +1200, Noah Gift wrote:
1. Different versions of Python conflict with previous versions of
console scripts. Take paste for example.
I don't understand what you mean.
Sorry that was a bit curt.
One of the problems with Pylons installation and virtualenv is that
you might have previously installed paste script, and that could have
been triggered to a specific version of paste like this:
#!/vol/apps/python-2.5.1_64/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'PasteScript==1.7.3','console_scripts','paster'
__requires__ = 'PasteScript==1.7.3'
import sys
from pkg_resources import load_entry_point
sys.exit(
load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
if you do a which paste, you will see this:
csh#ng...@naseberry# which paste
/usr/bin/paste
What this means is the you could really want some theoretical new
version of the paste script in your virtualenv, or in a standard
python site-packages directory, and or a different version of python,
say 2.6, but the name of the script hides which actual version you
want and it specifically chooses one version of Python. There isn't
that good of a solution to solve this, but easy_install itself, seems
to do the right thing, but appending the actual python version to
the script name.
I think the idea way is that one bootstrap could work across all
version of python so that python versions wouldn't need to be appended
to the script name. Additionally, it could be useful, but I have no
idea how this would work, to have the bootstrap dynamically pick the
right version of itself that it needs to run in a given context.
2. The entry point mechanism IIRC recursively scans the site-packages
directory and loads up the system path with eggs. This is too
expensive of an operation for the current environment I work in.
The scan is not recursive; only files that are actually *on* sys.path are
scanned: i.e., either an .egg that is directly on sys.path, or an .egg-info
in a directory that is directly on sys.path.
I could be wrong, as this is from memory, but I believe that each egg
inside of site-packages gets seperately injected in sys.path. I
suppose this is the only way to deal with package versions cleanly. I
have noticed that whole process is reasonably expensive.
Now, it's possible that some application you are using does such a scan
explicitly; I'm just noting that merely querying or loading entry points
doesn't cause any recursive scans, and it most definitely does not add
anything new to sys.path, unless the entry point to be loaded has declared
an additional dependency that's *not* on sys.path yet.
3. There doesn't seem to be a clean way to inject user specific
environment details to the console script.
I often need the ability to alter the sys.path in a user specific way
for the entry point without needing to mess up the global sys.path
permanently.
I don't understand what you mean here, either.
In film enviroments the whole way we work is by toggling between
different enviroments. A developer, artists, etc, will need to work
on a specific shot in a movie, and they also need to toggle between
films, etc. Each of these enviroments is different because different
code is being used. Keeping all possible combinations of python
environments in your global environment can make the python interpeter
grind to halt because of the way Python itself recursives over paths
looking for files. The idea scenario, at least for my current
environment, is for sys.path to inject a custom path or N paths at
the actual run time of the command line tool, because it limits the
scanning Python needs to do. Here is an example of my quick and dirty
hack to make IPython run quicker:
if __name__ == __main__:
import sys
sys.path = [
'/vol/apps/python-2.5.1_64/lib/python2.5',
'/vol/apps/python-2.5.1_64/lib/python2.5/site-packages',
'/vol/apps/python-2.5.1_64/lib/python2.5/lib-dynload/',
'/vol/filmstudio/linux64/lib/python2.5/site-packages/Genshi-0.4.4-py2.5.egg',
'/vol/filmstudio/linux64/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-linux-x86_64.egg',
'/vol/filmstudio/linux64/lib/python2.5/site-packages/lxml-2.1.2-py2.5-linux-x86_64.egg',
'/vol/filmstudio/linux64/lib/python2.5/site-packages/nose-0.10.4-py2.5.egg',
]
import IPython.Shell
IPython.Shell.start().mainloop()
On one hand, IPython is now very snappy and runs in a useable fashion.
The problem now, is that all flexibility is lost to dynamically add
more things based on the users environment. Perhaps by having the
boostrap script look at some custom environmental variable to inject
one more path, but just for that boostrap, in that given context,
Re: [Distutils] Adding entry points into Distutils ?
At 10:38 AM 5/7/2009 -0400, Doug Hellmann wrote: pip installs my scripts into a virtualenv without any issue and without using entry points, AFAICT. I guess if we move to requiring entry points and disallowing simple script distribution I'll need to find another way to package virtualenvwrapper. Since it's a bash script, it doesn't have entry points. I've been using setuptools to package it so it can be installed via easy_install, since it is a Python development tool. Setuptools still supports classic scripts, and I don't see any reason to remove that support. People do package non-Python scripts with their projects, after all. easy_install basically examines such scripts to see if they end with .py, have a #! line with 'python' in it, or are valid Python source code. If any of the above are true, it makes a Python script wrapper, otherwise it assumes the script is some other language and leaves it alone. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 7, 2009, at 7:28 PM, P.J. Eby wrote: At 10:38 AM 5/7/2009 -0400, Doug Hellmann wrote: pip installs my scripts into a virtualenv without any issue and without using entry points, AFAICT. I guess if we move to requiring entry points and disallowing simple script distribution I'll need to find another way to package virtualenvwrapper. Since it's a bash script, it doesn't have entry points. I've been using setuptools to package it so it can be installed via easy_install, since it is a Python development tool. Setuptools still supports classic scripts, and I don't see any reason to remove that support. People do package non-Python scripts with their projects, after all. easy_install basically examines such scripts to see if they end with .py, have a #! line with 'python' in it, or are valid Python source code. If any of the above are true, it makes a Python script wrapper, otherwise it assumes the script is some other language and leaves it alone. That would do what I want. I only brought it up because another poster in this thread mentioned eliminating scripts an requiring console apps to use entry points. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 01:14 PM 5/7/2009 -0700, Wheat wrote: But then having both 'scripts' and 'entry_points/console_scripts' is less than perfect since it introduces (mostly) unneccessary TIMTOWTDI. I interpret 'scripts' as meaning non-Python scripts, so there is still only one obvious way to do each thing. ;-) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Paul Moore wrote: That works but a lot of Unix users have in the past objected to having '.py' in the name. So install a symlink from hello - hello.py. -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 5, 2009, at 10:50 PM, P.J. Eby wrote: At 12:03 PM 5/6/2009 +1000, Ben Finney wrote: I don't see any advantage, in the context of this discussion, to having an additional, incompatible naming for full-path-to-a-class. Setuptools doesn't limit an entry point to being a class, function, or other top-level name within a module. It can be a method of a class, or an attribute of an attribute. The ':' removes any ambiguity as to which part of the name is the module, and which parts are attributes within that module. Is that level of complexity useful in practice? I can understand how it came to be implemented, but is it actually used by any projects? Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote: On May 5, 2009, at 10:50 PM, P.J. Eby wrote: At 12:03 PM 5/6/2009 +1000, Ben Finney wrote: I don't see any advantage, in the context of this discussion, to having an additional, incompatible naming for full-path-to-a-class. Setuptools doesn't limit an entry point to being a class, function, or other top-level name within a module. It can be a method of a class, or an attribute of an attribute. The ':' removes any ambiguity as to which part of the name is the module, and which parts are attributes within that module. Is that level of complexity useful in practice? I can understand how it came to be implemented, but is it actually used by any projects? I use it; I'm not sure who else does. The particular use case I have (and that's most likely to be shared) is that the calling app or framework wants a callable or function, but the providing app or library implements that callable as a classmethod for convenience. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Doug Hellmann wrote: On May 6, 2009, at 1:46 PM, P.J. Eby wrote: At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote: On May 5, 2009, at 10:50 PM, P.J. Eby wrote: At 12:03 PM 5/6/2009 +1000, Ben Finney wrote: I don't see any advantage, in the context of this discussion, to having an additional, incompatible naming for full-path-to-a-class. Setuptools doesn't limit an entry point to being a class, function, or other top-level name within a module. It can be a method of a class, or an attribute of an attribute. The ':' removes any ambiguity as to which part of the name is the module, and which parts are attributes within that module. Is that level of complexity useful in practice? I can understand how it came to be implemented, but is it actually used by any projects? I use it; I'm not sure who else does. The particular use case I have (and that's most likely to be shared) is that the calling app or framework wants a callable or function, but the providing app or library implements that callable as a classmethod for convenience. That's pretty much what I expected. It feels a little messy to have plugins exposing internals like that but not so much so that I propose we don't allow it. The : syntax seems like the right way to go. I'd be tempted to call this an edge-case. You should be able to expose the internal detail you'd need via a module scope alias for the particular case. That seems easier than providing a whole new notion. Hanno ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hanno Schlichting wrote: Doug Hellmann wrote: On May 6, 2009, at 1:46 PM, P.J. Eby wrote: At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote: On May 5, 2009, at 10:50 PM, P.J. Eby wrote: At 12:03 PM 5/6/2009 +1000, Ben Finney wrote: I don't see any advantage, in the context of this discussion, to having an additional, incompatible naming for full-path-to-a-class. Setuptools doesn't limit an entry point to being a class, function, or other top-level name within a module. It can be a method of a class, or an attribute of an attribute. The ':' removes any ambiguity as to which part of the name is the module, and which parts are attributes within that module. Is that level of complexity useful in practice? I can understand how it came to be implemented, but is it actually used by any projects? I use it; I'm not sure who else does. The particular use case I have (and that's most likely to be shared) is that the calling app or framework wants a callable or function, but the providing app or library implements that callable as a classmethod for convenience. That's pretty much what I expected. It feels a little messy to have plugins exposing internals like that but not so much so that I propose we don't allow it. The : syntax seems like the right way to go. I'd be tempted to call this an edge-case. You should be able to expose the internal detail you'd need via a module scope alias for the particular case. That seems easier than providing a whole new notion. I'm actually a big fan of the ':', because it makes explicit the difference between the import and the named thing, even for module-scoped names. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKAde9+gerLs4ltQ4RAgzJAJ0RgbDdXFFW/O/mcK3u6BCKOiBW3QCfUXs3 S0lgBewN4w5PqIHBilft29Y= =G7sv -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 08:28 PM 5/6/2009 +0200, Hanno Schlichting wrote: Doug Hellmann wrote: On May 6, 2009, at 1:46 PM, P.J. Eby wrote: At 10:59 AM 5/6/2009 -0400, Doug Hellmann wrote: On May 5, 2009, at 10:50 PM, P.J. Eby wrote: At 12:03 PM 5/6/2009 +1000, Ben Finney wrote: I don't see any advantage, in the context of this discussion, to having an additional, incompatible naming for full-path-to-a-class. Setuptools doesn't limit an entry point to being a class, function, or other top-level name within a module. It can be a method of a class, or an attribute of an attribute. The ':' removes any ambiguity as to which part of the name is the module, and which parts are attributes within that module. Is that level of complexity useful in practice? I can understand how it came to be implemented, but is it actually used by any projects? I use it; I'm not sure who else does. The particular use case I have (and that's most likely to be shared) is that the calling app or framework wants a callable or function, but the providing app or library implements that callable as a classmethod for convenience. That's pretty much what I expected. It feels a little messy to have plugins exposing internals like that but not so much so that I propose we don't allow it. The : syntax seems like the right way to go. I'd be tempted to call this an edge-case. You should be able to expose the internal detail you'd need via a module scope alias for the particular case. That seems easier than providing a whole new notion. Ah, now you reminded me of a forgotten detail: the specific use case was that I had a library with a Command class that was widely used as a base class; simply adding a classmethod to that base class allowed all the existing classes to be used as plugins without any change to their modules' source code: the entry points just always reference the classmethod. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Wed, May 6, 2009 at 1:32 PM, Tres Seaver tsea...@palladion.com wrote:
I'd be tempted to call this an edge-case. You should be able to expose
the internal detail you'd need via a module scope alias for the
particular case. That seems easier than providing a whole new notion.
I'm actually a big fan of the ':', because it makes explicit the
difference between the import and the named thing, even for
module-scoped names.
Yeah, I also like it primarily for clarity. Also you can provide better
error messages when it fails, matching up the error against the intention.
In some contexts I also extend this syntax to do something like:
module, path = name.split(':', 1)
obj = eval(path, load_module(module).__dict__)
I don't propose that we do eval for plugins, but it's a nice side-effect of
the syntax that it's possible to add in other contexts. Also I don't think
there's any strong precedence for purely dot notation, loading objects from
strings is something that's always done ad hoc, and the only widely used
library I know of that people use for this is Setuptools (indirectly through
entry points).
--
Ian Bicking | http://blog.ianbicking.org
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Ian Bicking wrote: On Wed, May 6, 2009 at 1:32 PM, Tres Seaver tsea...@palladion.com mailto:tsea...@palladion.com wrote: I'm actually a big fan of the ':', because it makes explicit the difference between the import and the named thing, even for module-scoped names. Yeah, I also like it primarily for clarity. Also you can provide better error messages when it fails, matching up the error against the intention. [...] Also I don't think there's any strong precedence for purely dot notation, loading objects from strings is something that's always done ad hoc, and the only widely used library I know of that people use for this is Setuptools (indirectly through entry points). The one I and others use in quite many places is zope.dottedname [1] (the actual function is at [2]). This one doesn't make any difference between modules and attributes. It has no zope dependency but happens to be used throughout the entire Zope / Plone stack. Hanno [1] http://pypi.python.org/pypi/zope.dottedname [2] http://svn.zope.org/zope.dottedname/trunk/src/zope/dottedname/resolve.py?view=markup ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Tue, May 5, 2009 at 1:57 AM, Ian Bicking i...@colorstudy.com wrote: Not strong, but I have a few issues with how they are currently defined: * There's the issue of activated and unactivated eggs, of course, but I guess that will be moot since there's no activation with just distutils? Yes * There's no idea of explicitly enabling an entry point, simply installing a package makes the entry point show up. Implicit plugins make me uncomfortable. I don't see entry points as plugins, but rather the registering of a given piece of code, under a unique name. If you add explicit enabling, who will do it ? the package that has the entry point ? The applications that consumes them ? The way I see entry points is potential plugins, an application can decide to consume, and turn into a real plugin when it uses it. And an entry point that would be disabled is an entry point that is not used from the application A point of view, but might be used in the application B. (unless I misunderstood the concept of group) So enabling/disabling an entry point and keeping track of the activation state should be done by the host application ihmo. * There's not much in the way of entry point documentation built into the system. The __doc__ of the entry point objects is one way, but there's no way to document entry point groups. very good point, -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
2009/5/5 P.J. Eby p...@telecommunity.com: At 06:57 PM 5/4/2009 -0500, Ian Bicking wrote: * I'm uncomfortable with the way entry points are scanned. I haven't looked close enough to back it up with numbers, but I think there's a noticeable performance degradation when the number of installed packages becomes large. (Given the algorithm this would be expected.) It's linear in the number of entry_points.txt files, yes, but in most apps it should occur at most once, since pkg_resources has a single WorkingSet object holding Distribution objects which cache their entry point data upon first access. There are all sorts of ways you could make different tradeoffs, but this particular set of tradeoffs was optimized for a single-application environment, rather than a massive global shared site-packages where there are plugins for every application on the system. It was also optimized for the zipimport case, because you can tell whether a project has entry points from its cached zip directory, that's needed at startup anyhow. Would it make sense then to maintain an global index of all entry points, that would be updated upon installation / uninstallation (if it's added like we wrote in PEP 376) rather than scanning the paths everytime ? we could have one index file per site-package-like directory, and discover index files rather than all directories / zip files. Extra paths added in sys.path would be omited but I don't see it as a problem -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 5, 2009, at 4:49 AM, Tarek Ziadé wrote: On Tue, May 5, 2009 at 1:57 AM, Ian Bicking i...@colorstudy.com wrote: Not strong, but I have a few issues with how they are currently defined: * There's the issue of activated and unactivated eggs, of course, but I guess that will be moot since there's no activation with just distutils? Yes * There's no idea of explicitly enabling an entry point, simply installing a package makes the entry point show up. Implicit plugins make me uncomfortable. I don't see entry points as plugins, but rather the registering of a given piece of code, under a unique name. I don't understand that. I thought the purpose of entry points was to register code such as plugins so that applications didn't have to be manually configured. I think I'm with Ian on that one: Explicit is better than implicit. If I have to turn on the plugin, then what benefit does an entry point registry give me? I could just as easily provide that information to the application directly. If you add explicit enabling, who will do it ? the package that has the entry point ? The applications that consumes them ? The user who wants the application to consume the plugin. The way I see entry points is potential plugins, an application can decide to consume, and turn into a real plugin when it uses it. And an entry point that would be disabled is an entry point that is not used from the application A point of view, but might be used in the application B. But if it's not being used by A, why should A see it at all? Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Tue, May 5, 2009 at 1:57 PM, Doug Hellmann doug.hellm...@gmail.com wrote: If I have to turn on the plugin, then what benefit does an entry point registry give me? I don't understand this sentence, since you say later that you want the user to manually turn a plugin on for an application to soncume it. If you add explicit enabling, who will do it ? the package that has the entry point ? The applications that consumes them ? The user who wants the application to consume the plugin. I am confused with the role of this man in the middle. In my mind there are plugins on one side, and host applications that consumes them if they wish on they other side. Do you have a use case we can share for mutual comprehension ? And an entry point that would be disabled is an entry point that is not used from the application A point of view, but might be used in the application B. But if it's not being used by A, why should A see it at all? A, B or any app can browse all entry points. Entry points are defined by the (group, name) couple. Basically if I want to create a pluggable feature called myfeature, the group will be myfeature and plugins that implement that feature will register themselves under that group. Then my application will browse and consume entry points for the group myfeature and do whatever they want with them. In pseudo code: plugins = iter_entry_points(group=myfeature) So if A doesn't need the plugins that are under the group myfeature, it will just ignore the entry points that are in this group. e.g. ignore the group. Maybe A will consume entry_points that are under another group. But I have never browsed *all* entry points from an application. I think the best practice for entry points is to use the most explicit group names possible, but having plugins that can be consumed by several applications is a win ihmo. For instance, if I need to write a specific extensible installation script, I'll probably see if I can consume zc.buildout recipes through the zc.buildout.recipe group. Doug -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 5, 2009, at 8:15 AM, Tarek Ziadé wrote: On Tue, May 5, 2009 at 1:57 PM, Doug Hellmann doug.hellm...@gmail.com wrote: If I have to turn on the plugin, then what benefit does an entry point registry give me? I don't understand this sentence, since you say later that you want the user to manually turn a plugin on for an application to soncume it. I want each application to be configured separately, rather than having a global registry of plugins. So having a plugin registry *library* in stdlib may make sense (so that apps can build their registry databases in a consistent way), but automatically registering entry points just because a package is installed does not. If you add explicit enabling, who will do it ? the package that has the entry point ? The applications that consumes them ? The user who wants the application to consume the plugin. I am confused with the role of this man in the middle. In my mind there are plugins on one side, and host applications that consumes them if they wish on they other side. Do you have a use case we can share for mutual comprehension ? I think we have a different view about how the plugins should work. It sounds like you're advocating a model where all plugins are registered globally, an application can search the global registry for plugins based on categories, and then some administrator enables/ disables them locally for each app. I don't want new functionality available to an application just because someone has permission to install a package somewhere in the PYTHONPATH. I would rather have plugins added to an app through an explicit configuration step of some sort. So if A doesn't need the plugins that are under the group myfeature, it will just ignore the entry points that are in this group. e.g. ignore the group. Maybe A will consume entry_points that are under another group. But I have never browsed *all* entry points from an application. That depends on how the database of entry points is maintained, but I'll grant that point. I think the best practice for entry points is to use the most explicit group names possible, but having plugins that can be consumed by several applications is a win ihmo. For instance, if I need to write a specific extensible installation script, I'll probably see if I can consume zc.buildout recipes through the zc.buildout.recipe group. Doug -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 5, 2009, at 9:33 AM, Tarek Ziadé wrote: On Tue, May 5, 2009 at 2:41 PM, Doug Hellmann doug.hellm...@gmail.com wrote: I am confused with the role of this man in the middle. In my mind there are plugins on one side, and host applications that consumes them if they wish on they other side. Do you have a use case we can share for mutual comprehension ? I think we have a different view about how the plugins should work. It sounds like you're advocating a model where all plugins are registered globally, an application can search the global registry for plugins based on categories, and then some administrator enables/disables them locally for each app. I don't want new functionality available to an application just because someone has permission to install a package somewhere in the PYTHONPATH. I would rather have plugins added to an app through an explicit configuration step of some sort. That is basically how host applications are dealing with entry points: an explicit configuration. The implicit part is just happening when you look for the plugins. Let's take a real example : I am working on a program called Atomisator which can be extended through plugins. For example people can provide a plugin to read data that are located somewhere (like a rss feed), and return a sequence of entries. So in atomisator, I have a configuration file where I decide which plugin to use to read my data: [atomisator] reader = rss When reading it, Atomisator knows that it needs the rss plugin, and will browse in entry points with a specific group called atomisator.rss. If it finds it, it uses it, otherwise it throws an exception. (Install it!) That allows people to create their own plugins in separate packages, and use them by tweaking the configuration file. The only think that entry point provided here is an automatic registration at installation time of the rss plugins, so my Atomisatior application can discover then load it at run time. I don't think I understand the difference between the step you're calling discover, scanning the registry, and actually loading the plugin. Does discovering the plugin involve importing any of its code? So in your way of seeing thing, you'd rather see this registration mechanism at the application level, but the you need to provide specific installation instructions for people that want to add plugins. e.g. put your package in this /plugins directory for example. Sort of. I see the installation and configuration of entry points as 2 steps: 1. some variation of python setup.py install 2. update the configuration of app to tell it where to find the plugin Some applications may provide automated tools for step 2, and I could see great benefit to making the loading and registration of entry points part of the standard library, as long as the registration is maintained per-application instead of site-wide. I am advocating that the entry point mechanism is handy because it relies on an existing mechanism : install your package and it allows plugin sharing amongst applications. How then do I install a package but *not* have it available for an application? Suppose, for example, that I have several copies of a web app installed for different customers, but I want to prevent some of them from using an advanced plugin of some sort. But I see the caveats you are explaining, and I understand them now; So, what if we didn't have these entry points installed globally when the package is installed, but just a configuration file in the host application level to point them ? a configuration file that reunites all entry points an application uses. For the Atomisator example: [atomisator.reader] rss = somepackage.somemodule:MyPluginClass Yes! We can figure out the exact spelling, but we're talking about the same thing now. If we use dotted notation all the way (somepackage.somemodule.MyPluginClass) then it's simple to just import the thing directly. This would let the application consume the plugins pointed by this configuration file and this would remove the implicit part you don't like. I'd be very happy with such a plugin system on my side. If you follow the model of the logging module and provide an easy load plugins from an ini file, then all python apps have the feature of making it easy to load plugins, developers have a standard API to code to, and administrators don't have to worry about plugins sneaking into an app. And this would fit I think in Distutils needs since we can configure it through three levels of configuration files distutils.cfg, pydistutils.cfg and setup.cfg That sounds good. Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 08:41 AM 5/5/2009 -0400, Doug Hellmann wrote: I don't want new functionality available to an application just because someone has permission to install a package somewhere in the PYTHONPATH. I would rather have plugins added to an app through an explicit configuration step of some sort. Note that this is not incompatible with entry points; an application can simply treat entry points as a list of *available* plugins, rather than as a list of *active* plugins. Chandler, for example, does this for user-level plugins. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 5, 2009, at 10:38 AM, P.J. Eby wrote: At 08:41 AM 5/5/2009 -0400, Doug Hellmann wrote: I don't want new functionality available to an application just because someone has permission to install a package somewhere in the PYTHONPATH. I would rather have plugins added to an app through an explicit configuration step of some sort. Note that this is not incompatible with entry points; an application can simply treat entry points as a list of *available* plugins, rather than as a list of *active* plugins. Chandler, for example, does this for user-level plugins. That's good. I wasn't sure if registering an entry point caused it to be automatically loaded in an app that expressed interest in it, or if there was a step the app could take to verify that it was desirable code before doing any imports. Earlier messages in this thread made me think there was no protection from a registered plugin. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Tue, May 5, 2009 at 4:29 PM, Doug Hellmann doug.hellm...@gmail.com wrote
I don't think I understand the difference between the step you're calling
discover, scanning the registry, and actually loading the plugin. Does
discovering the plugin involve importing any of its code?
No, like Phillip said somewhere in the thread.
The discovery pseudo-code is:
entry_points = []
for path in paths:
if path is egg-info:
entry_points.append(load('entry_points.txt'))
Then each entry point is just a string that locates the plugins
(package.modul.class for example)
The real import occurs only explicitely when you do entry_point.load()
a configuration file that reunites all entry points an application
uses. For the Atomisator example:
[atomisator.reader]
rss = somepackage.somemodule:MyPluginClass
Yes! We can figure out the exact spelling, but we're talking about the same
thing now. If we use dotted notation all the way
(somepackage.somemodule.MyPluginClass) then it's simple to just import the
thing directly.
I think is simpler with the somepackage.somemodule:MyPluginClass notation
This is how setuptools does roughly:
parts = somepackage.somemodule:MyPluginClass.split(':')
module = __import__(parts[0])
plugin = getattr(module, parts[1])
And this would fit I think in Distutils needs since we can configure
it through three levels of configuration files
distutils.cfg, pydistutils.cfg and setup.cfg
That sounds good.
The only caveat I see though, is that the host app has to know the
exact location
of each plugin in the code of the third party app, whereas entry points provide
this information through the discovery API.
I could leave with both notation I believe:
location = somepackage.somemodule:MyPluginClass
*or*
location = iter_entry_points('atomisator.reader', 'rss')
--
Tarek Ziadé | http://ziade.org
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
2009/5/5 Sebastien Binet seb.bi...@gmail.com: hi, That allows people to create their own plugins in separate packages, and use them by tweaking the configuration file. The only think that entry point provided here is an automatic registration at installation time of the rss plugins, so my Atomisatior application can discover then load it at run time. So in your way of seeing thing, you'd rather see this registration mechanism at the application level, but the you need to provide specific installation instructions for people that want to add plugins. e.g. put your package in this /plugins directory for example. wouldn't this be tackled by providing a plugin-discoverer plugin ? kind of like the new import hooks of pep-302 [1] where you have module finders separated from module loaders, or like this library I have been using in my field [2] That would be better to use these hooks rather than __import__ I believe, but I don't think it changes the problem (eg locate the plugin) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On May 5, 2009, at 11:05 AM, Tarek Ziadé wrote: On Tue, May 5, 2009 at 4:29 PM, Doug Hellmann doug.hellm...@gmail.com wrote a configuration file that reunites all entry points an application uses. For the Atomisator example: [atomisator.reader] rss = somepackage.somemodule:MyPluginClass Yes! We can figure out the exact spelling, but we're talking about the same thing now. If we use dotted notation all the way (somepackage.somemodule.MyPluginClass) then it's simple to just import the thing directly. I think is simpler with the somepackage.somemodule:MyPluginClass notation Good point, I was remembering the import syntax incorrectly. And this would fit I think in Distutils needs since we can configure it through three levels of configuration files distutils.cfg, pydistutils.cfg and setup.cfg That sounds good. The only caveat I see though, is that the host app has to know the exact location of each plugin in the code of the third party app, whereas entry points provide this information through the discovery API. True. On the other hand, that encourages standard locations like mylib.yourapp.entry_point and mylib.anotherapp.entry_point. I think most of my concerns about the global registry are taken care of by the fact that discovering a plugin doesn't involve any imports of unknown code. I still prefer per-app, explicit configuration of entry points, and think we could build a system to support that. I would like to see *some* variation of this in the standard library, though, because I have several uses for it. Doug ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Tue, May 05, 2009 at 05:05:28PM +0200, Tarek Ziadé wrote: On Tue, May 5, 2009 at 4:29 PM, Doug Hellmann doug.hellm...@gmail.com wrote a configuration file that reunites all entry points an application uses. For the Atomisator example: [atomisator.reader] rss = somepackage.somemodule:MyPluginClass [...] And this would fit I think in Distutils needs since we can configure it through three levels of configuration files distutils.cfg, pydistutils.cfg and setup.cfg That sounds good. So there would be a configuration file for each application that needs it? I like this a lot more then the global entry-point registry too (it avoids name collisions for entry points too). But how can a python setup.py install know where to find this configuration file to add it's plugin? Or should this be an explicit manual step? It might be nice to have a --register-plugins option to the install command though if possible. Something else to keep into account is the FHS, I can imagine GNU/Linux distributions would want to place a configuration file somewhere else, like in /etc/PROJECT.conf instead of /usr/lib/pythonX.Y/site-packages/PROJECT.egg-info/plugin_registry (pathnames are fairly random examples). The only thing I can think of is somehow having a file in .egg-info telling you where the plugin configuration file is, just like was proposed for all types of data files earlier on this list. But I think by now I'm taking this too far and the install command of distutils should not be able to register plugins for random projects automatically. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 09:27 PM 5/5/2009 +0100, Floris Bruynooghe wrote: But how can a python setup.py install know where to find this configuration file to add it's plugin? It doesn't. The whole point of having two stages -- discovery and activation -- is that discovery is an automatic side-effect of installation, whereas activation can be controlled by an application-specific policy or configuration file. Entry points (as currently implemented) are discovery; the application then determines which entry points to actually load or invoke, using its own configuration (or command-line options, or menu selections, or automatic policy, or whatever else). ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Tarek Ziadé ziade.ta...@gmail.com writes:
I think is simpler with the somepackage.somemodule:MyPluginClass notation
This is how setuptools does roughly:
parts = somepackage.somemodule:MyPluginClass.split(':')
[…]
Using the standard import notation is no more difficult:
parts = somepackage.somemodule.MyPluginClass.rsplit('.', 1)
parts
['somepackage.somemodule', 'MyPluginClass']
versus:
parts = somepackage.somemodule:MyPluginClass.split(':', 1)
parts
['somepackage.somemodule', 'MyPluginClass']
I don't see any advantage, in the context of this discussion, to having
an additional, incompatible naming for full-path-to-a-class.
--
\ “The lift is being fixed for the day. During that time we |
`\regret that you will be unbearable.” —hotel, Bucharest |
_o__) |
Ben Finney
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 12:03 PM 5/6/2009 +1000, Ben Finney wrote: I don't see any advantage, in the context of this discussion, to having an additional, incompatible naming for full-path-to-a-class. Setuptools doesn't limit an entry point to being a class, function, or other top-level name within a module. It can be a method of a class, or an attribute of an attribute. The ':' removes any ambiguity as to which part of the name is the module, and which parts are attributes within that module. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
On Tue, 5 May 2009 01:46:21 +0200, Tarek Ziadé ziade.ta...@gmail.com wrote: Hello I am making a separate email for this topic to make sure no one misses it. There are *many* benefits of adding entry points into Distutils. In fact, adding a plugin system in there, will help make the commands more extendable and therefore will help us in the long term to remove things out of Distutils. So any strong opinion against them ? but what does it provide that we don't already get with the site-packages directory? and .PTH? isn't there already a plug-in system for packages in python? David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
At 06:57 PM 5/4/2009 -0500, Ian Bicking wrote: * I'm uncomfortable with the way entry points are scanned. I haven't looked close enough to back it up with numbers, but I think there's a noticeable performance degradation when the number of installed packages becomes large. (Given the algorithm this would be expected.) It's linear in the number of entry_points.txt files, yes, but in most apps it should occur at most once, since pkg_resources has a single WorkingSet object holding Distribution objects which cache their entry point data upon first access. There are all sorts of ways you could make different tradeoffs, but this particular set of tradeoffs was optimized for a single-application environment, rather than a massive global shared site-packages where there are plugins for every application on the system. It was also optimized for the zipimport case, because you can tell whether a project has entry points from its cached zip directory, that's needed at startup anyhow. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Adding entry points into Distutils ?
Tarek Ziadé ziade.ta...@gmail.com writes: There are *many* benefits of adding entry points into Distutils. In fact, adding a plugin system in there, will help make the commands more extendable and therefore will help us in the long term to remove things out of Distutils. I don't see what those advantages are. From what I can see, commands still need to be implemented in separate programs in order to be used as commands. So any strong opinion against them ? Until I understand what they do, I don't know what objections I might have :-) -- \“Only the educated are free.” —Epictetus | `\ | _o__) | Ben Finney ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
