[Distutils] Easy install, making Traditional PYTHONPATH-based Installation work better
I'be been learning about setuptools and eggs and am very pleased. There is one area where I'd like to see a small tweak. In: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations four options are presented for installing outside of Python installation. Naturally, I prefer the the last option, which the documentation recommends against. :) Here's why. An important use case to me is the ability for a developer to work in a development sandbox without affecting anything outside the sandbox. The solution needs to be cross platform. The Administrator Installation option doesn't work for me because it requires modifying the Python setup. Neither the Mac OS X User Installation nor the 'Creating a Virtual Python' options work for me because they are not cross platform. I also find the Virtual Python approach a bit heavy handed. In playing a bit, I find the Traditional PYTHONPATH-based Installation works pretty well except for the problem of having to set the path manually when invoking scripts. This problem could be addressed by having the generated scripts include the custom path settings. I recommend adding two new options to the easy_install section of the configuration file: script_path A list of paths, separated by commas to be added to sys.path by generated scripts. script_pth_files A list of pth files, separated by commas to be added be processed by script files. Generated scripts would include code to manipulate sys.path based on this information. So, for example, in a workspace with a 'lib' directory containing eggs, I'd have: [easy_install] site_dirs = '/here/lib' script_path = '/here/lib' script_pth_files = '/here/lib/easy-install.pth' The script_pth_files option is needed, I think, to avoid hardcoding the setuptools egg path in the generated scripts. Note that it could be argued that the site_dirs option should be enough and that generated scripts should simply reflect this option. Thoughts? For now, I can create a wrapper around easy_install that does custom script generation, which could serve as a prototype for the proposed change. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Easy install, making Traditional PYTHONPATH-based Installation work better
At 10:52 AM 2/5/2006 -0500, Jim Fulton wrote: I'be been learning about setuptools and eggs and am very pleased. There is one area where I'd like to see a small tweak. In: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations four options are presented for installing outside of Python installation. Naturally, I prefer the the last option, which the documentation recommends against. :) Here's why. An important use case to me is the ability for a developer to work in a development sandbox without affecting anything outside the sandbox. The solution needs to be cross platform. The Administrator Installation option doesn't work for me because it requires modifying the Python setup. Neither the Mac OS X User Installation nor the 'Creating a Virtual Python' options work for me because they are not cross platform. I also find the Virtual Python approach a bit heavy handed. In playing a bit, I find the Traditional PYTHONPATH-based Installation works pretty well except for the problem of having to set the path manually when invoking scripts. This problem could be addressed by having the generated scripts include the custom path settings. Unfortunately, that's probably harder than you think. The reason I advise against the PYTHONPATH-based approach is that it uses a sneaky hack to get site directory processing to work properly (i.e., as desired). When you use the PYTHONPATH-based approach, it only works if you put the setuptools egg directly on the PYTHONPATH at *Python startup*, because setuptools includes its own version of the 'site' module that bypasses the standard Python 'site' module, so that it can scan .pth files that are *already on* PYTHONPATH. This means that no Python code embedded in the scripts is going to affect this. If you want to use PYTHONPATH, you have to set PYTHONPATH in the environment. By the time the Python script itself is running, sys.path has already been manipulated such that it's no longer possible to process .pth files in a way that gives them precedence over system-installed packages. Now, if you don't care about the .pth files and just want to use whatever the script's require() picks up, then it's not a problem. However, since your post talked about .pth and site-dirs, I assumed that means you want to process that stuff. For now, I can create a wrapper around easy_install that does custom script generation, which could serve as a prototype for the proposed change. Try it out, and see if you can get a privately installed package to *override* a system-installed one, without setting PYTHONPATH in the environment prior to Python startup. I'll be surprised and intrigued if you can find a way. (*Adding* packages this way is of course easy; it's overriding system-installed packages that's the problem, because you have to get the .pth stuff in *front* of site-packages in sys.path, and that normally doesn't happen. It only happens with setuptools because of its hacked 'site.py'.) If you don't care about .pth files, then you can simply install the desired eggs or egg-links to the same directory as the scripts, and PYTHONPATH only needs to include the setuptools egg. It's either that, or perhaps put some bootstrap code in every script to locate setuptools if it's not already on sys.path. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Easy install, making Traditional PYTHONPATH-based Installation work better
At 02:16 PM 2/5/2006 -0500, Jim Fulton wrote: It's too bad setuptools isn't in the standard library. If it was, I'd probably just stick all my eggs and scripts in the same place and be done ith it. :) It's still tempting. Note that it's only actually necessary for *pkg_resources* to be importable; you don't have to have all of setuptools. If you installed pkg_resources.py to your target directory, your whole plan would work just fine. By the way, note that if people are developing from source, you probably want to use 'setup.py develop' rather than 'easy_install', since it allows you to edit the code and continue to run. The 'develop' command should work the same with all this, just make sure there's a pkg_resources.py in the target directory when you actually run your scripts. Indeed, given your requirements as I understand them, that should really be all you need. You don't need a custom site-dirs, you need only configure the installation directory (--install-dir) for easy_install and develop. Develop inherits configuration from easy_install, so a ~/.pydistutils.cfg like this: [easy_install] install_dir = wherever should do the trick as long as wherever/pkg_resources.py exists. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
