Re: [Distutils] PEP 376 for Distutils
On Wed, Feb 25, 2009 at 4:16 PM, P.J. Eby p...@telecommunity.com wrote: I wonder why they don't just use the sitewide distutils.cfg file, which would let them configure user-installed packages to go to somewhere else, e.g.: [install] prefix = /usr/local (And of course the build tools could specify different options.) Some systems do this. I've found this problematic with virtualenv, as people get weird results because they don't realize this configuration file exists. Also there's no way currently in distutils to unset a value, so if something like prefix is given explicitly there's no way to indicate it should be calculated (you just have to give an explicit value for --prefix). -- Ian Bicking | http://blog.ianbicking.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
2009/2/25 P.J. Eby p...@telecommunity.com: At 10:03 PM 2/25/2009 +, Floris Bruynooghe wrote: It's interesting to point out what seems to be planned for Debian: http://lists.debian.org/debian-devel/2009/02/msg00431.html Quoting just the relevant part: Local installation path --- When installing Python modules using distutils, the resulting files end up in the same location wether they are installed by a Debian package, or by a local user or administrator, unless the installation path is overwritten on the command line. Compare this with most software based on autoconf, where an explicit prefix has to be provided for the packaging, while the default install installs into /usr/local. For new Python versions packaged in Debian this will change so that an installation into /usr (not /usr/local) requires an extra option to distutils install command (--install-layout=deb). To avoid breaking the packaging of existing code the distutils install command for 2.4 and 2.5 will just accept this option and ignore it. I wonder why they don't just use the sitewide distutils.cfg file, which would let them configure user-installed packages to go to somewhere else, e.g.: [install] prefix = /usr/local (And of course the build tools could specify different options.) There might be an issue with this while building python itself. I remember having issues failing to build python because I had [install] home=/home/flub in my .pydistutils.cfg. When python is building and installing it's standard library it uses distutils and having that in my configuration file messed everything up and the builds failed (this was both when building the upstream tarball as well as rebuilding the debian package). This is all IIRC however, I'd have to try this again to remind me of the exact issues (thinking about it now this is probably because python's makefile doesn't invoke setup.py with enough options to be sure to overwrite the configuration file). So you'd have to build-conflict against python, which would be annoying at best, but you also need python to build python these days (again IIRC, might be possible to get round this) so you're stuck. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
Tarek Ziadé wrote: ok so far, from the whole dicussion it seems that everyone agrees that the Python version is superfluous in the .egg-info files, so I'll update the PEP for this point. I'll also start to write more details about uninstallation As P.J.Eby pointed out the importance of PEP 262, could all the egg-info directories be also put into a subdirectory called install-db as described in that PEP. As proposed by P.J.Eby in PEP262 under open issues, there could/should be one of these for each directory where packages are installed. So to repeat the initial example for zlib, it should look like this: zlib/ install_db/ zlib-2.5.2.egg-info/ PKG-INFO MANIFEST RECORD The name of the install_db directory would not be that important, e.g. it could e.g. be called egg-info and the .egg-info extension from the zlib egg-info directory name could be stripped, making the parsing of the listdir() call even faster 8-) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
2009/2/25 Joachim König h...@online.de: Tarek Ziadé wrote: ok so far, from the whole dicussion it seems that everyone agrees that the Python version is superfluous in the .egg-info files, so I'll update the PEP for this point. I'll also start to write more details about uninstallation As P.J.Eby pointed out the importance of PEP 262, could all the egg-info directories be also put into a subdirectory called install-db as described in that PEP. As proposed by P.J.Eby in PEP262 under open issues, there could/should be one of these for each directory where packages are installed. So to repeat the initial example for zlib, it should look like this: zlib/ install_db/ zlib-2.5.2.egg-info/ PKG-INFO MANIFEST RECORD The name of the install_db directory would not be that important, e.g. it could e.g. be called egg-info and the .egg-info extension from the zlib egg-info directory name could be stripped, making the parsing of the listdir() call even faster 8-) Let's call it install-db then, like suggested in PEP 262 : this will ensure that it cannot be a package/module name because of the -. I doubt anyone is calling a module or a package install_db out there but it's safer that way Tarek ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
At 09:16 AM 2/25/2009 +0100, Joachim König wrote: Tarek Ziadé wrote: ok so far, from the whole dicussion it seems that everyone agrees that the Python version is superfluous in the .egg-info files, so I'll update the PEP for this point. I'll also start to write more details about uninstallation As P.J.Eby pointed out the importance of PEP 262, I pointed out the importance its *requirements* and *criteria* -- not its implementation. There's a big difference. could all the egg-info directories be also put into a subdirectory called install-db as described in that PEP. As proposed by P.J.Eby in PEP262 under open issues, there could/should be one of these for each directory where packages are installed. So to repeat the initial example for zlib, it should look like this: zlib/ install_db/ zlib-2.5.2.egg-info/ PKG-INFO MANIFEST RECORD The name of the install_db directory would not be that important, e.g. it could e.g. be called egg-info and the .egg-info extension from the zlib egg-info directory name could be stripped, making the parsing of the listdir() call even faster 8-) I'm -1 on this, as it breaks backward compatibility with setuptools. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
On Tue, Feb 24, 2009 at 11:48:13AM -0800, Andrew Straw wrote: Ronald Oussoren wrote: What about another interoperability hook for system packages: specify a file that a (system) package manager can include into the egg-info directory (or egg-file) to tell setuptools/pip that this egg is managed by the system and hence shouldn't be removed by setuptools/pip. Which such a file the user of python package tool could be warned if he tries to uninstall an egg that's owned by the system, instead of invoking the wrath of a sysadmin by removing such files. But that is already implemented via file/dir permissions. Not all systems are managed by experienced sysadmins and on many single user systems `sudo' is all to easy to invoke. By your reasoning, we should also have something which warns users not to install to the system directory. These ideas are a duplication of functionality -- this functionality is implemented by the disabling write permissions of non-sysadmins into system directories. As pointed out by PJE simply removing the RECORD file should do the trick. Or do you propose users put some stuff into their system directories not managed by their package managers and other stuff managed by the package managers? It's interesting to point out what seems to be planned for Debian: http://lists.debian.org/debian-devel/2009/02/msg00431.html Quoting just the relevant part: Local installation path --- When installing Python modules using distutils, the resulting files end up in the same location wether they are installed by a Debian package, or by a local user or administrator, unless the installation path is overwritten on the command line. Compare this with most software based on autoconf, where an explicit prefix has to be provided for the packaging, while the default install installs into /usr/local. For new Python versions packaged in Debian this will change so that an installation into /usr (not /usr/local) requires an extra option to distutils install command (--install-layout=deb). To avoid breaking the packaging of existing code the distutils install command for 2.4 and 2.5 will just accept this option and ignore it. For the majority of packages we won't see changes in the packaging, provided that the python packaging helpers can find the files in the right location and move it to the expected target path. A second issue raised by developers was the clash of modules and extensions installed by a local python installation (with default prefix /usr/local) with the modules provided by Debian packages (/usr/local/lib/pythonX.Y/site-packages shared by the patched system python and the locally installed python. To avoid this clash the directory `site-packages' should be renamed to `dist-packages' in both locations: - /usr/lib/pythonX.Y/dist-packages (installation location for code packaged for Debian) - /usr/local/lib/pythonX.Y/dist-packages (installation location for locally installed code using distutils install without options). The path /usr/lib/pythonX.Y/site-packages is not found on sys.path anymore. About the name: Discussed this with Barry Warsaw and Martin v. Loewis, and we came to the conclusion that using the same directory name for both locations would be the most consistent way. This change should make the request to conditionalize the inclusion of /usr/local/lib/pythonX.Y/site-packages into sys.path obsolete. If needed we can provide a symlink /usr/lib/pythonX.Y/site-packages pointing to dist-packages. This should address the concern of sysadmins making mistakes when adding and removing python distributions (packages/modules). In other words, maybe it's not python's problem but the OS distribution's problem. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
At 10:03 PM 2/25/2009 +, Floris Bruynooghe wrote: It's interesting to point out what seems to be planned for Debian: http://lists.debian.org/debian-devel/2009/02/msg00431.html Quoting just the relevant part: Local installation path --- When installing Python modules using distutils, the resulting files end up in the same location wether they are installed by a Debian package, or by a local user or administrator, unless the installation path is overwritten on the command line. Compare this with most software based on autoconf, where an explicit prefix has to be provided for the packaging, while the default install installs into /usr/local. For new Python versions packaged in Debian this will change so that an installation into /usr (not /usr/local) requires an extra option to distutils install command (--install-layout=deb). To avoid breaking the packaging of existing code the distutils install command for 2.4 and 2.5 will just accept this option and ignore it. I wonder why they don't just use the sitewide distutils.cfg file, which would let them configure user-installed packages to go to somewhere else, e.g.: [install] prefix = /usr/local (And of course the build tools could specify different options.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
Tarek Ziadé wrote: If you look at install_egg_info, it will add the Python version http://svn.python.org/projects/python/trunk/Lib/distutils/command/install_egg_info.py I am not sure either this should be kept. I don't see the rationale either, since sys.version is known at runtime, it seems superfluous. Maybe it should be deprecated. I having the same problem with the version : since it is already located in PKG-INFO, there's no need to have it in the folder name; So maybe the final version could be: - zlib - zlib.egg-info/ PKG-INFO MANIFEST RECORD And while we are at it: could the egg-info directory be put somewhere else (as a configuration/command line option)? It's meta information about the packages and not python code. And when doing a simple 'ls' on the site-packages folders I woul like to see the packages only and not 2 entries per package. An other option could be to put the egg-info dir into the package itself, e.g. zlib/ __init__.py egg-info/ PKG-INFO MANIFEST RECORD ... P.J. Eby wrote about the encoding of package and python version inside the egg-info directory name: It's there so pkg_resources doesn't need to read the file in order to locate an available version of the package, just the listdir() that it was doing anyway. In other words, it's a performance optimization, and a pretty major one when you're building an Environment object to look for available package versions. Cuts down on a LOT of file opens in the case where you have a ton of .egg-info's in site-packages, like on a Linux system with Python 2.5. If the location of the egg-info dir and the encoding of the python version and package version are only there for performance optimization reasons I'd suggest to really decouple optimization from file naming and having a caching directory that can compute an optimized representation once when an out-of-date situation is detected (e.g. when new packages are installed) making the optimization even faster, e.g. by having a file in suitable format instead of calling os.listdir() and iterating over the result. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
Philip wrote: When a distutils package does it. I'm not positive, but if 'pip' supports namespace packages without using .pth files, then it has to use a shared __init__ also. And in the long run, easy_install will do this too. So, the uninstallation code should simply not remove file(s) that are referenced by more than one manifest in the target directory -- a relatively simple, future-proof safeguard, that doesn't require any specific knowledge of namespace packages per se. Sounds good. Although, it requires scanning the files again which is not optimal, but the last point of this mail might be an idea to adress this. 2009/2/24 Joachim König h...@online.de: An other option could be to put the egg-info dir into the package itself, e.g. zlib/ __init__.py egg-info/ PKG-INFO MANIFEST RECORD ... This would require setuptools and pip to change the way they look for the packages, but if the functions to work with this are located in Python, they will be able to use the same bits which could be great. If the location of the egg-info dir and the encoding of the python version and package version are only there for performance optimization reasons I'd suggest to really decouple optimization from file naming and having a caching directory that can compute an optimized representation once when an out-of-date situation is detected (e.g. when new packages are installed) making the optimization even faster, e.g. by having a file in suitable format instead of calling os.listdir() and iterating over the result. Indeed. Having an index file would make things a whole lot simpler. I am wondering then if this is not an evolution of the .pth files. Although I find that having as many .pth file as we want is not robust. It make things slow down when you have too many of them. So, I'd be in favor of a new, unique, optimized, index file, with a set of functions located in pkgutil to read and write in it This index file could also index the namespace information, in order to speed up the work needed to uninstall a package that shares a namespace. Regards Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
At 01:33 PM 2/24/2009 +0100, Tarek Ziadé wrote: Philip wrote: So, the uninstallation code should simply not remove file(s) that are referenced by more than one manifest in the target directory -- a relatively simple, future-proof safeguard, that doesn't require any specific knowledge of namespace packages per se. Sounds good. Although, it requires scanning the files again which is not optimal, but the last point of this mail might be an idea to adress this. Uninstallation isn't exactly a performance-critical activity. It's trivial to read targetdir/*/RECORD and build up a dictionary of file-package and package-file links; this is Python, after all. ;-) 2009/2/24 Joachim König h...@online.de: An other option could be to put the egg-info dir into the package itself, e.g. zlib/ __init__.py egg-info/ PKG-INFO MANIFEST RECORD ... This would require setuptools and pip to change the way they look for the packages, More precisely, it would require pkg_resources to become ridiculously slow, by massively increasing the number of directories that need to be read at runtime to determine what packages are present. Indeed. Having an index file would make things a whole lot simpler. For *whom*? Certainly not for system packaging tools (rpm, deb, et al). A design goal should be to allow system packaging tools to install a static file footprint: i.e., independent files with predefined content, and no post-processing steps. You can't do that with a shared file, which is why setuptools uses a .pth hack to install namespace packages when building packages for rpm et al. I am wondering then if this is not an evolution of the .pth files. Although I find that having as many .pth file as we want is not robust. It make things slow down when you have too many of them. So, I'd be in favor of a new, unique, optimized, index file, with a set of functions located in pkgutil to read and write in it This index file could also index the namespace information, in order to speed up the work needed to uninstall a package that shares a namespace. So, .pth files are bad... let's make more? In fact, let's make a new thing that'll have its own, new bugs? So that we can have uninstalls that take only 1/10th of a second instead of 1/2 a second? The standard to beat in this area, I believe, is PEP 262. At the very least, if you're making any major changes in direction from that PEP, the rationale for those differences should be documented. (And consolidated index files are explicitly rejected by that PEP, with good reason.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
P.J. Eby schrieb: At 09:39 AM 2/24/2009 +0100, Joachim König wrote: could the egg-info directory be put somewhere else (as a configuration/command line option)? No, since it's used to identify the installed location of the code that goes with it, ala PEP 262. In other words, sys.path is its own installation database. But PEP262 suggests prefix/lib/pythonversion/install-db/ for the location and the open issues there state (among other things): PJE suggests the installation database be potentially present on every directory in sys.path, So PEP262 implies to me that we either would have one install-db/ directory on prefix/lib/pythonversion/ or one on each directory of sys.path, e.g. also one on site-packages. This would really solve my problem, having only one entry more per directory of sys.path instead of having one directory per packages at the locations I generally look for the code. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
On 24 Feb, 2009, at 13:33, Tarek Ziadé wrote: 2009/2/24 Joachim König h...@online.de: An other option could be to put the egg-info dir into the package itself, e.g. zlib/ __init__.py egg-info/ PKG-INFO MANIFEST RECORD ... This would require setuptools and pip to change the way they look for the packages, Not only that, it also requires that the name of the distribution is the same as the name of (one of) its python packages. I have several projects where this correspondence is not present. but if the functions to work with this are located in Python, they will be able to use the same bits which could be great. If the location of the egg-info dir and the encoding of the python version and package version are only there for performance optimization reasons I'd suggest to really decouple optimization from file naming and having a caching directory that can compute an optimized representation once when an out-of-date situation is detected (e.g. when new packages are installed) making the optimization even faster, e.g. by having a file in suitable format instead of calling os.listdir() and iterating over the result. Indeed. Having an index file would make things a whole lot simpler. I don't see how this would make thing easier. An index file introduces another concept and requires care to ensure that it doesn't get out of date. Ronald smime.p7s Description: S/MIME cryptographic signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
On Tue, Feb 24, 2009 at 04:39:24PM +0100, Ronald Oussoren wrote: On 24 Feb, 2009, at 13:33, Tarek Ziadé wrote: 2009/2/24 Joachim König h...@online.de: An other option could be to put the egg-info dir into the package itself, e.g. zlib/ __init__.py egg-info/ PKG-INFO MANIFEST RECORD ... This would require setuptools and pip to change the way they look for the packages, Not only that, it also requires that the name of the distribution is the same as the name of (one of) its python packages. I have several projects where this correspondence is not present. It also requires that a distribution installs a package. What about single-module packages? Indeed. Having an index file would make things a whole lot simpler. I don't see how this would make thing easier. An index file introduces another concept and requires care to ensure that it doesn't get out of date. An index file should probably a be private optimisation of a tool and not be required or standard. See also P. Eby's arguments about this. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
At 04:45 PM 2/24/2009 +0100, Ronald Oussoren wrote: What about another interoperability hook for system packages: specify a file that a (system) package manager can include into the egg-info directory (or egg-file) to tell setuptools/pip that this egg is managed by the system and hence shouldn't be removed by setuptools/pip. Which such a file the user of python package tool could be warned if he tries to uninstall an egg that's owned by the system, instead of invoking the wrath of a sysadmin by removing such files. The absence of a RECORD file could suffice for that, I think. Any file that isn't positively listed in *exactly* one RECORD file shouldn't be removed. I'm not super-set on that position, just noting that the absence of a listing is sufficient to detect something that shouldn't (as opposed to mustn't) be deleted. The only contradicting use case is the deletion of packages that were installed via an older distutils version. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
At 06:21 PM 2/24/2009 +, Floris Bruynooghe wrote: On Mon, Feb 23, 2009 at 09:53:17PM -0500, P.J. Eby wrote: So, the uninstallation code should simply not remove file(s) that are referenced by more than one manifest in the target directory -- a relatively simple, future-proof safeguard, that doesn't require any specific knowledge of namespace packages per se. But it does require that the uninstall reads all *.egg-info/RECORD files in a site directory when uninstalling. That's slow, but I guess that's fair enough. You're going to need to look at all the egg-info dirs anyway, at least if you're planning to verify whether another installed package depends on the package you're removing. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
2009/2/24 P.J. Eby p...@telecommunity.com: At 09:39 AM 2/24/2009 +0100, Joachim König wrote: could the egg-info directory be put somewhere else (as a configuration/command line option)? No, since it's used to identify the installed location of the code that goes with it, ala PEP 262. In other words, sys.path is its own installation database. If the location of the egg-info dir and the encoding of the python version and package version are only there for performance optimization reasons I'd suggest to really decouple optimization from file naming and having a caching directory that can compute an optimized representation once when an out-of-date situation is detected (e.g. when new packages are installed) making the optimization even faster, e.g. by having a file in suitable format instead of calling os.listdir() and iterating over the result. That would introduce the possibility of the cache being out of date. See PEP 262. (By the way, the Python version info isn't needed for this optimization; only the package version.) ok so far, from the whole dicussion it seems that everyone agrees that the Python version is superfluous in the .egg-info files, so I'll update the PEP for this point. I'll also start to write more details about uninstallation ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
At 02:57 PM 2/24/2009 -0500, P.J. Eby wrote: At 04:45 PM 2/24/2009 +0100, Ronald Oussoren wrote: What about another interoperability hook for system packages: specify a file that a (system) package manager can include into the egg-info directory (or egg-file) to tell setuptools/pip that this egg is managed by the system and hence shouldn't be removed by setuptools/pip. Which such a file the user of python package tool could be warned if he tries to uninstall an egg that's owned by the system, instead of invoking the wrath of a sysadmin by removing such files. The absence of a RECORD file could suffice for that, I think. Any file that isn't positively listed in *exactly* one RECORD file shouldn't be removed. I'm not super-set on that position, just noting that the absence of a listing is sufficient to detect something that shouldn't (as opposed to mustn't) be deleted. The only contradicting use case is the deletion of packages that were installed via an older distutils version. Hm. This is also relevant for *install* operations... which should not overwrite (without warning) a file that's not listed in any RECORD files, even if it has the exact same content. (As that would mean a later uninstall would silently delete it.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
Hello Tarek
On Sun, Feb 22, 2009 at 03:37:50AM +0100, Tarek Ziadé wrote:
I have started a PEP for Distutils, I would like to work out for Pycon.
http://svn.python.org/projects/peps/trunk/pep-0376.txt
Looks quite nice so far IMHO. Here some early feedback...
Back to our `zlib` example, we will have::
- zlib
- zlib-2.5.2-py2.6.egg-info/
PKG-INFO
MANIFEST
RECORD
What is the rationale for the `-2.5.2-py2.6' part in the name of the
.egg-info directory? I'm not saying I'm against it, rather that I'd
like to see a rationale for why it is there and why it is different
from the current distutils bit (doesn't add the python version, only
package version).
Related to this how is the behaviour of pkgutil.get_egg_info() defined
in respect to this? It mentions that all `pkg_name*.egg-info' files
are looked for and None is returned if none is found. But no word on
what happens if more then one is found. (Should that be an error?
Guess it depends on the anser to the previous paragraph.)
Let's use it over our `zlib` example::
from pkgutil import get_egg_info, get_metadata,
get_egg_info_file
get_egg_info('zlib')
'/opt/local/lib/python2.6/site-packages/zlib-2.5.2-py2.6.egg-info'
metadata = get_metadata('zlib')
metadata.version
'2.5.2'
from distutils.dist import EGG_INFO_FILES
get_metadata('zlib', EGG_INFO_FILES.manifest).read()
some
...
files
I think there's a typo in the last call here. Should that not be
`get_egg_info_file('zlib', EGG_INFO_FILES.manifest).read()'?
Finally about the install and uninstall script. Is this going to
attempt to address the namespace packages problem or will this be left
for a later PEP? AIUI if it doesn't (which seems reasonable,
distutils never could do this) but setuptools does install such a
package then distutils will happily try to remove it, using the
.egg-info/RECORD file. But this might then mess up any uninstall
attempt from setuptools since a shared __init__.py might have been
removed.
Hope this was helpful in some way.
Regards
Floris
--
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
On Tue, Feb 24, 2009 at 12:13 AM, Floris Bruynooghe
floris.bruynoo...@gmail.com wrote:
Hello Tarek
On Sun, Feb 22, 2009 at 03:37:50AM +0100, Tarek Ziadé wrote:
I have started a PEP for Distutils, I would like to work out for Pycon.
http://svn.python.org/projects/peps/trunk/pep-0376.txt
Looks quite nice so far IMHO. Here some early feedback...
Back to our `zlib` example, we will have::
- zlib
- zlib-2.5.2-py2.6.egg-info/
PKG-INFO
MANIFEST
RECORD
What is the rationale for the `-2.5.2-py2.6' part in the name of the
.egg-info directory? I'm not saying I'm against it, rather that I'd
like to see a rationale for why it is there and why it is different
from the current distutils bit (doesn't add the python version, only
package version).
If you look at install_egg_info, it will add the Python version
http://svn.python.org/projects/python/trunk/Lib/distutils/command/install_egg_info.py
I am not sure either this should be kept. I don't see the rationale
either, since
sys.version is known at runtime, it seems superfluous.
Maybe it should be deprecated.
I having the same problem with the version : since it is already
located in PKG-INFO,
there's no need to have it in the folder name;
So maybe the final version could be:
- zlib
- zlib.egg-info/
PKG-INFO
MANIFEST
RECORD
Related to this how is the behaviour of pkgutil.get_egg_info() defined
in respect to this? It mentions that all `pkg_name*.egg-info' files
are looked for and None is returned if none is found. But no word on
what happens if more then one is found. (Should that be an error?
Guess it depends on the anser to the previous paragraph.)
In Distutils, there should be one egg-info per package. (While
setuptools multi-package
options allows to activate/deactivate various versions of the same package).
That said, I don't think this PEP should address multiple versions issues.
I'll add some details about this.
Let's use it over our `zlib` example::
from pkgutil import get_egg_info, get_metadata,
get_egg_info_file
get_egg_info('zlib')
'/opt/local/lib/python2.6/site-packages/zlib-2.5.2-py2.6.egg-info'
metadata = get_metadata('zlib')
metadata.version
'2.5.2'
from distutils.dist import EGG_INFO_FILES
get_metadata('zlib', EGG_INFO_FILES.manifest).read()
some
...
files
I think there's a typo in the last call here. Should that not be
`get_egg_info_file('zlib', EGG_INFO_FILES.manifest).read()'?
right,
Finally about the install and uninstall script. Is this going to
attempt to address the namespace packages problem or will this be left
for a later PEP? AIUI if it doesn't (which seems reasonable,
distutils never could do this) but setuptools does install such a
package then distutils will happily try to remove it, using the
.egg-info/RECORD file. But this might then mess up any uninstall
attempt from setuptools since a shared __init__.py might have been
removed.
Right. I think we can try to address namespace packages in another
PEP, because as far as I
can see, the namespace package boilerplate in setuptools is using
pkgutil.extend_path which
fixes __path__ variables on the fly. So I can't think of a case where
a static __init__.py file
will be shared, thus removed.
Hope this was helpful in some way.
Thanks for the feedback Floris, I'll update the PEP accordingly
Regards
Floris
--
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
___
Distutils-SIG maillist - distutils-...@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
--
Tarek Ziadé | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 376 for Distutils
At 01:50 AM 2/24/2009 +0100, Tarek Ziadé wrote: I having the same problem with the version : since it is already located in PKG-INFO, there's no need to have it in the folder name; It's there so pkg_resources doesn't need to read the file in order to locate an available version of the package, just the listdir() that it was doing anyway. In other words, it's a performance optimization, and a pretty major one when you're building an Environment object to look for available package versions. Cuts down on a LOT of file opens in the case where you have a ton of .egg-info's in site-packages, like on a Linux system with Python 2.5. Now, it is supported to leave that information out, but that's really intended only for working with development packages/checkouts, where you don't have so many of them on sys.path at any one time that the extra open is a big deal. In Distutils, there should be one egg-info per package. (While setuptools multi-package options allows to activate/deactivate various versions of the same package). That said, I don't think this PEP should address multiple versions issues. Note that sys.path could have more than one version of the same package on it, in two different directories. (Granted, one would take precedence over the other, but that should still be explicit.) PEP, because as far as I can see, the namespace package boilerplate in setuptools is using pkgutil.extend_path which fixes __path__ variables on the fly. So I can't think of a case where a static __init__.py file will be shared, thus removed. When a distutils package does it. I'm not positive, but if 'pip' supports namespace packages without using .pth files, then it has to use a shared __init__ also. And in the long run, easy_install will do this too. So, the uninstallation code should simply not remove file(s) that are referenced by more than one manifest in the target directory -- a relatively simple, future-proof safeguard, that doesn't require any specific knowledge of namespace packages per se. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
