Re: [Python-Dev] PEP: per user site-packages directory
I don't really understand what all this has to do with per user site-packages. Note that the motivation for having per user site-packages was to: * address a common request by Python extension package users, * get rid off the hackery done by setuptools in order to provide this. As such the PEP can also be seen as an effort to enable code cleanup *before* adding e.g. pkg_resources to the stdlib. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 22 2008) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 On 2008-01-21 16:06, Nick Coghlan wrote: Steve Holden wrote: Christian Heimes wrote: Steve Holden wrote: Maybe once we get easy_install as a part of the core (so there's no need to find and run ez_setup.py to start with) things will start to improve. This is an issue the whole developer community needs to take seriously if we are interested in increasing take-up. setuptools and easy_install won't be included in Python 2.6 and 3.0: http://www.python.org/dev/peps/pep-0365/ Yes, and yet another release (two releases) will go out without easy access to the functionality in Pypi. PEP 365 is a good start, but Pypi loses much of its point until new Python users get access to it out of the box. I also appreciate that resource limitations are standing in the way of setuptools' inclusion (is there something I can do about that?) Just to hammer the point home, however ... Have another look at the rationale given in PEP 365 - it isn't the resourcing to do the work that's a problem, but the relatively slow release cycle of the core. By including pkg_resources in the core (with the addition of access to pure Python modules and packages on PyPI), we would get a simple, stable base for Python packaging to work from, and put users a single standard command away from the more advanced (but also more volatile) features of easy_install and friends. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
At 04:42 PM 1/22/2008 +0100, M.-A. Lemburg wrote: I don't really understand what all this has to do with per user site-packages. Note that the motivation for having per user site-packages was to: * address a common request by Python extension package users, * get rid off the hackery done by setuptools in order to provide this. Setuptools doesn't do any hackery for per-user site-packages, although its documentation does explain how to set up such a thing if you want it: http://peak.telecommunity.com/DevCenter/EasyInstall#administrator-installation http://peak.telecommunity.com/DevCenter/EasyInstall#mac-os-x-user-installation Meanwhile, note that having per-user site-packages directories doesn't eliminate the need to be able to have PYTHONPATH directories treated as site directories, which is hasn't been discussed at all. As such the PEP can also be seen as an effort to enable code cleanup *before* adding e.g. pkg_resources to the stdlib. Code cleanup of what? There's nothing in pkg_resources that would change for per-user site package directories, since pkg_resources doesn't do any installation work. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
M.-A. Lemburg wrote: I don't really understand what all this has to do with per user site-packages. Note that the motivation for having per user site-packages was to: * address a common request by Python extension package users, * get rid off the hackery done by setuptools in order to provide this. As such the PEP can also be seen as an effort to enable code cleanup *before* adding e.g. pkg_resources to the stdlib. I guess it's a typical case of while you on it, can you fix the rest of the world, too?. People see that I invest a fair share of time into the PEP. So they hope I'm going to solve loosely connected issues, too. Personally I'm not going to solve every security issue with my PEP. I suggest that all the people, who were interested in security, come together and write another PEP. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
-On [20080120 19:34], Christian Heimes ([EMAIL PROTECTED]) wrote: Most Python developers should the meaning of ~. Should I replace ~ with $HOME for those who don't have as much experience with Unix as we? The problem is that ~ is an expansion character. It expands the contents of $HOME. If HOME is set to /usr/local, using ~ will expand to /usr/local. This behaviour is undefined by POSIX, if HOME is null/empty than expanding ~ is undefined. If it is just ~ then it will expand to the contents of HOME, if ~ is followed by a string then it will do a getpwnam() call on the string and expand that to the initial working directory of said login name string. HOME is POSIX-mandated to be '[t]he pathname of the user's home directory'. -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Hope is the last refuge for mad men and dreamers... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
-On [20080116 07:15], Martin v. Löwis ([EMAIL PROTECTED]) wrote: I don't understand why they *have* to do that. I can believe they do that as they don't know better - but why can't they use the Python interpreter already available on the system, and just install additional packages in their home directory? Good question. I guess there just isn't a clear and precise document available to point them at. I'll be glad to (help) write such an article. For that, I think the requirements need to be much more explicit. Yeah, sorry, have not had the time available to verify all cases Christian reported versus what I can think of. Impending surgery tends to distract the mind a bit. ;) -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Where does the world end if the enternal optimist loses faith..? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Martin v. Löwis wrote: Right now Python faces a lot of problems in the webhosting world because it's tedious to set up and maintain for the webhosting user since they often have to compile and install their own Python in their home directory. I don't understand why they *have* to do that. I can believe they do that as they don't know better - but why can't they use the Python interpreter already available on the system, and just install additional packages in their home directory? One possible reason is that the hosting vendor doesn't provide a sufficiently up-to-date Python release for certain site requirements. I think this is extremely important due to the proliferation of Python now more and more as a choice for webapp development. For that, I think the requirements need to be much more explicit. True. But IMHO Python is still seen as a difficult language to package. We haven't reached CPAN levels of ease-of-use ye. Yes, I *do* know that CPAN isn't perfect, and I think easy_install is a great boon to Python users, but CPAN does a great job 98% of the time. Maybe once we get easy_install as a part of the core (so there's no need to find and run ez_setup.py to start with) things will start to improve. This is an issue the whole developer community needs to take seriously if we are interested in increasing take-up. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Steve Holden wrote: Maybe once we get easy_install as a part of the core (so there's no need to find and run ez_setup.py to start with) things will start to improve. This is an issue the whole developer community needs to take seriously if we are interested in increasing take-up. setuptools and easy_install won't be included in Python 2.6 and 3.0: http://www.python.org/dev/peps/pep-0365/ Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Christian Heimes wrote: Steve Holden wrote: Maybe once we get easy_install as a part of the core (so there's no need to find and run ez_setup.py to start with) things will start to improve. This is an issue the whole developer community needs to take seriously if we are interested in increasing take-up. setuptools and easy_install won't be included in Python 2.6 and 3.0: http://www.python.org/dev/peps/pep-0365/ Yes, and yet another release (two releases) will go out without easy access to the functionality in Pypi. PEP 365 is a good start, but Pypi loses much of its point until new Python users get access to it out of the box. I also appreciate that resource limitations are standing in the way of setuptools' inclusion (is there something I can do about that?) Just to hammer the point home, however ... The easy_install utility from setuptools is probably the biggest potential recruiter for Python. It makes the language accessible to the majority of users who are interested only in acquiring new functionality, and not at all in which language that functionality is implemented in. Installation of the latest/most appropriate version of something then becomes a simple recipe without command switches regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Steve Holden wrote: Christian Heimes wrote: Steve Holden wrote: Maybe once we get easy_install as a part of the core (so there's no need to find and run ez_setup.py to start with) things will start to improve. This is an issue the whole developer community needs to take seriously if we are interested in increasing take-up. setuptools and easy_install won't be included in Python 2.6 and 3.0: http://www.python.org/dev/peps/pep-0365/ Yes, and yet another release (two releases) will go out without easy access to the functionality in Pypi. PEP 365 is a good start, but Pypi loses much of its point until new Python users get access to it out of the box. I also appreciate that resource limitations are standing in the way of setuptools' inclusion (is there something I can do about that?) Just to hammer the point home, however ... Have another look at the rationale given in PEP 365 - it isn't the resourcing to do the work that's a problem, but the relatively slow release cycle of the core. By including pkg_resources in the core (with the addition of access to pure Python modules and packages on PyPI), we would get a simple, stable base for Python packaging to work from, and put users a single standard command away from the more advanced (but also more volatile) features of easy_install and friends. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Op zondag 13-01-2008 om 10:45 uur [tijdzone +0100], schreef Christian Heimes: Gregory P. Smith wrote: My main suggestion was going to be the ability to turn it off as you already mentioned. However, please consider leaving it off by default to avoid problems for installed python scripts importing user supplied code. For shared hosting environments where this becomes really useful users can easily add the -s (or whatever flag is chosen) to their programs themselves. I don't know what that'd mean on windows where #! lines don't exist. Yet another file extension to imply the flag (yuck)? A .cmd wrapper script to run python with the flag (ugh)? So you prefer to make the per use site-package directory an opt-in option? I prefer it as an opt-out option. It's enabled by default, unless the user disables the feature with -s. One possible issue with enabling it by default could be that less-experienced users can accidentally break the system configuration tools written in python used by several linux distros (Ubuntu, Gentoo, Fedora/Red Hat, ...). There should be a way for distro developers to make sure the users local 'site-packages' is *not* used when running those tools. I'd rather have to set/uncomment an environment variable on my system than having 100 normal users break their systems accidentally... ;-) -- Jan Claeys ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Op zaterdag 12-01-2008 om 00:27 uur [tijdzone +0100], schreef Christian Heimes: Specification = [...] user configuration directory Usually the parent directory of the user site directory. It's meant for Python version specific data like config files. Windows: %APPDATA%/Python/Python26 Mac: ~/Library/Python/2.6 Unix: ~/.local/lib/python2.6 What do you mean by configuration directory? IMHO configuration files on linux/unix should go into ~/.python2.6 or ~/.config/python2.6 or something like that? (I would never look in a directory named 'lib' for configuration files.) -- Jan Claeys ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Jan Claeys wrote: What do you mean by configuration directory? IMHO configuration files on linux/unix should go into ~/.python2.6 or ~/.config/python2.6 or something like that? It's already renamed in the PEP: http://www.python.org/dev/peps/pep-0370/#specification Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
-On [20080116 07:15], Oleg Broytmann ([EMAIL PROTECTED]) wrote: The site only mentions $HOME/.local/share, there is no $HOME/.local/bin at the site. As was mentioned earlier in the thread. Only $HOME/.local/share was in the Free Desktop specification. The bin is something that got introduced in this thread. It makes sense to stick to hier(7) for this sort of thing. Although the entire concept of a $HOME/.local was new to me as well and is not mandated at all in POSIX or other circles aside from, apparently, the Free Desktop folks. Pendantic note: ~ is an expansion character, the correct variable to talk about is HOME (see IEEE Std 1003.1, 2004 section 2.5.3 and 2.6.1). -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ Peace comes from within. Do not seek it without. - Buddha ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Jan Claeys wrote: There should be a way for distro developers to make sure the users local 'site-packages' is *not* used when running those tools. There is an option. Those tools should use the -E and -s argument: #!/usr/bin/env python -E -s Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Op zondag 20-01-2008 om 18:01 uur [tijdzone +0100], schreef Christian Heimes: Jan Claeys wrote: What do you mean by configuration directory? IMHO configuration files on linux/unix should go into ~/.python2.6 or ~/.config/python2.6 or something like that? It's already renamed in the PEP: http://www.python.org/dev/peps/pep-0370/#specification So this is stuff that should never be changed by the user? -- Jan Claeys ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Jeroen Ruigrok van der Werven wrote: Pendantic note: ~ is an expansion character, the correct variable to talk about is HOME (see IEEE Std 1003.1, 2004 section 2.5.3 and 2.6.1). Most Python developers should the meaning of ~. Should I replace ~ with $HOME for those who don't have as much experience with Unix as we? Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Jan Claeys wrote: So this is stuff that should never be changed by the user? ~/.local/lib/python2.6 has the same semantics as /usr/local/lib/python2.6 except it's a per user directory and not per machine. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Jeroen Ruigrok van der Werven wrote: One thing I miss in this PEP and discussion is the point of view from a webhosting company and webhosting user. If the webhoster upgrades his hosting software and Python get updated by a revision (say 2.5 to 2.6) this would in the current case mean that the webhosting user using per-user site-packages is out of luck due to the hard-wired (and apparently needed) version identifier. Or is that a wrong assumption on my side? The problem exists for almost every Python extension. 3rd party packages with C modules have to be compiled for every Python version. Even pure Python packages don't always work on a new version. You can't expect software to run under every version of FOO, not even the next future version of FOO where FOO is Python, Java, C#, Perl, PHP ... you name it. It's not the intention of the PEP to solve the problem. A web hoster can support multiple versions of Python at once. You can install every minor version of Python from 2.0 to 3.0 (8 versions) without major conflicts. A *good* web hoster should give an user several months up to a year to migrate from - say - 2.5 to 2.6. Most Linux distributions (except Redhat) have at least 2.4 and 2.5 in their repositories. Right now Python faces a lot of problems in the webhosting world because it's tedious to set up and maintain for the webhosting user since they often have to compile and install their own Python in their home directory. The per-user site-packages will help in that aspect a lot, I think, but we do need to detail some more use-cases and scenarios (patterns if you like) on how such deployments would work. I think this is extremely important due to the proliferation of Python now more and more as a choice for webapp development. Can you write up some use cases for us? I'm especially interested in use cases that are not covered by PEP 370. So far only a few problems come into my mind: * Installation extensions for mod_python, requires root permissions or full control over the Apache process anyway * Multiple versions of an extension. The problem is mostly covered by pkg_resources and setup tools. * Installing a newer version of an extension, covered by PEP 370. * Registering more search paths. Drop a pth file into the user site directory or use a usercustumize.py. * easy_install packages to the user site directory. Setuptools are not part of the Python stdlib. I'm confident that Phil is going to add the feature as soon as the feature is implemented for Python 2.6 What's missing? Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Mon, Jan 14, 2008 at 11:41:47PM +, Jon Ribbens wrote: It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. Me too. python-dev is the only place I have heard of ~/.local. I have been using Linux (different distributions), Solaris and FreeBSD for quite a long time (though I have never used GNOME/KDE/etc.) Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Jan 15, 2008 6:24 AM, Oleg Broytmann [EMAIL PROTECTED] wrote: On Mon, Jan 14, 2008 at 11:41:47PM +, Jon Ribbens wrote: It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. Me too. python-dev is the only place I have heard of ~/.local. I have been using Linux (different distributions), Solaris and FreeBSD for quite a long time (though I have never used GNOME/KDE/etc.) Never heard of it either, would be completely baffled if caught unawares by it in the wild. Has anyone consulted with the LSB or a cross-platform filesystem layout guide to see what the recommended best-practice is? -Kevin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Tue, Jan 15, 2008 at 01:21:46PM +0100, Christian Heimes wrote: It took me a while to find a reference to .local. It's part of the FreeDesktop.Org standards: http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html The site only mentions $HOME/.local/share, there is no $HOME/.local/bin at the site. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Kevin Jacobs [EMAIL PROTECTED] wrote: Never heard of it either, would be completely baffled if caught unawares by it in the wild. Has anyone consulted with the LSB or a cross-platform filesystem layout guide to see what the recommended best-practice is? It took me a while to find a reference to .local. It's part of the FreeDesktop.Org standards: http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Right now Python faces a lot of problems in the webhosting world because it's tedious to set up and maintain for the webhosting user since they often have to compile and install their own Python in their home directory. I don't understand why they *have* to do that. I can believe they do that as they don't know better - but why can't they use the Python interpreter already available on the system, and just install additional packages in their home directory? I think this is extremely important due to the proliferation of Python now more and more as a choice for webapp development. For that, I think the requirements need to be much more explicit. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Christian Heimes wrote: [...] PEP: XXX Title: Per user site-packages directory Version: $Revision$ Last-Modified: $Date$ Author: Christian Heimes christian(at)cheimes(dot)de Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 11-Jan-2008 Python-Version: 2.6, 3.0 Post-History: [...] user site directory A site directory inside the users' home directory. An user site directory is specific to a Python version. The path contains the version number (major and minor only). Windows: %APPDATA%/Python/Python26/site-packages Mac: ~/Library/Python/2.6/site-packages Unix: ~/.local/lib/python2.6/site-packages user configuration directory Usually the parent directory of the user site directory. It's meant for Python version specific data like config files. Windows: %APPDATA%/Python/Python26 Mac: ~/Library/Python/2.6 Unix: ~/.local/lib/python2.6 So if I'm using the --user option, where would scripts be installed? Would this be: Windows: %APPDATA%/Python/Python26/bin Mac: ~/Library/Python/2.6/bin Unix: ~/.local/lib/python2.6/bin I'd like to be able to switch between several versions of my user installation simply by changing a link. (On the Mac I'm doing this by relinking ~/Library/Python to different directories.) Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On 12:08 pm, [EMAIL PROTECTED] wrote: So if I'm using the --user option, where would scripts be installed? Would this be: Windows: %APPDATA%/Python/Python26/bin Mac: ~/Library/Python/2.6/bin Unix: ~/.local/lib/python2.6/bin I'd like to be able to switch between several versions of my user installation simply by changing a link. (On the Mac I'm doing this by relinking ~/Library/Python to different directories.) I think the relevant link to change here would be ~/.local. I have personally been using the ~/.local convention for a while, and I believe ~/.local/bin is where scripts should go. Python is not the only thing that can be locally installed, and the fact that it's ~/.local/lib/python2.6/site-packages suggests that ~/.local has the same layout as /usr (or /usr/local, for those who use that convention). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
[EMAIL PROTECTED] wrote: the standard directory? according to what? commented-out examples in some linux distribution? Yes ... :/ I should be more carefully when I use the word standard. But, now that I've told you what I think in more detail, unless you like my ideas and have specific questions, I will try to refrain from commenting further, lest I dig my own bike shed here :). I have no strong opinion on ~/bin or ~/.local/bin. I understand your point of view and I concur with some of your arguments. I'm making this discussion point easy for me. I let the others decide! :) I don't feel like wasting my time on a bike shed discussion unless it's painted in a different color than blue. I like blue ... *g* Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Jan 14, 2008, at 2:13 PM, [EMAIL PROTECTED] wrote: ~/bin comes from the convention of ./configure --prefix=$HOME, as autoconf suggests. This means users must have visible directories in their home folder named (among other things) bin, share, lib, src, sbin, man, and include. I find this ugly. I only find it slightly less ugly that Python will now hide its locally-installed library files from me but not its locally-installed executable scripts. ./configure --prefix=$HOME/.local will instead put things into ~/.local/bin, ~/.local/lib, etc. Now, I am aware that setup.py already has a special --home option, and it can be incompatibly changed, but I don't see a reason for this. Note that --home $HOME will currently put files into ~/lib, not ~/.local/ lib. (and --home $HOME/local will put scripts into ~/.local/bin, not ~/ bin). But, now that I've told you what I think in more detail, unless you like my ideas and have specific questions, I will try to refrain from commenting further, lest I dig my own bike shed here :). I feel pretty strongly that ~/bin should *not* be used. It makes sense to me that ~/.local would mirror /usr/local. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR4vA+XEjvBPtnXfVAQICEgP8CAuFVzZld1769uQKDWj2h4Y7x+besq5o 9bujNYJ6SayNac4u1jWLWmCIdBSuQU6/xNF6+ljpn5Pz4H/yRBl/HK2ibF2ksZwg quv23PHLvMnLju77FNKE5VclVJk3rBKpkpjmS/yXMcyfBwAccIDEJY+QUshtQzql 8mRZ4cEUP0I= =gepD -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Barry Warsaw wrote: I feel pretty strongly that ~/bin should *not* be used. It makes sense to me that ~/.local would mirror /usr/local. Two votes for ~/.local/bin and one undecided PEP author ... I'm changing the code to ~/.local/bin and I'm adding a new section to the PEP. Can I just submit the PEP or do I have to follow a procedure before I can add it to the PEP list? Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Jan 14, 2008 12:27 PM, Christian Heimes [EMAIL PROTECTED] wrote: Barry Warsaw wrote: I feel pretty strongly that ~/bin should *not* be used. It makes sense to me that ~/.local would mirror /usr/local. Two votes for ~/.local/bin and one undecided PEP author ... I'm changing the code to ~/.local/bin and I'm adding a new section to the PEP. Can I just submit the PEP or do I have to follow a procedure before I can add it to the PEP list? You can submit the PEP and update PEP 0 (note that each PEP occurs twice!) but you can't mark it accepted :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
The PEP is now available at http://www.python.org/dev/peps/pep-0370/. The reference implementation is in svn, too: svn+ssh://[EMAIL PROTECTED]/sandbox/trunk/pep370 Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On 2008-01-14 22:23, Christian Heimes wrote: The PEP is now available at http://www.python.org/dev/peps/pep-0370/. The reference implementation is in svn, too: svn+ssh://[EMAIL PROTECTED]/sandbox/trunk/pep370 Thanks for the effort, Christian. Much appreciated. Regarding the recent ~/bin vs. ~/.local/bin discussion: I usually maintain my ~/bin directories by hand and wouldn't want any application to install things in there automatically (and so far I haven't been using any application that does), so I'd be in favor of the ~/.local/bin dir. Note that users typically don't know which scripts are made available by a Python application and it's not always clear what functionality they provide, whether they can be trusted, include bugs, need to be run with extra care, etc, so IMHO making it a little harder to run them by accident is well warranted. Thanks again, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 14 2008) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Mon, Jan 14, 2008 at 03:07:20PM -0500, Barry Warsaw wrote: I feel pretty strongly that ~/bin should *not* be used. It makes sense to me that ~/.local would mirror /usr/local. It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Jan 14, 2008 6:41 PM, Jon Ribbens [EMAIL PROTECTED] wrote: It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. *raises hand* I have one, fwiw. -- Cheers, Leif ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
One thing I miss in this PEP and discussion is the point of view from a webhosting company and webhosting user. If the webhoster upgrades his hosting software and Python get updated by a revision (say 2.5 to 2.6) this would in the current case mean that the webhosting user using per-user site-packages is out of luck due to the hard-wired (and apparently needed) version identifier. Or is that a wrong assumption on my side? Right now Python faces a lot of problems in the webhosting world because it's tedious to set up and maintain for the webhosting user since they often have to compile and install their own Python in their home directory. The per-user site-packages will help in that aspect a lot, I think, but we do need to detail some more use-cases and scenarios (patterns if you like) on how such deployments would work. I think this is extremely important due to the proliferation of Python now more and more as a choice for webapp development. -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ The greatest of faults, I should say, is to be conscious of none... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Gregory P. Smith wrote: My main suggestion was going to be the ability to turn it off as you already mentioned. However, please consider leaving it off by default to avoid problems for installed python scripts importing user supplied code. For shared hosting environments where this becomes really useful users can easily add the -s (or whatever flag is chosen) to their programs themselves. I don't know what that'd mean on windows where #! lines don't exist. Yet another file extension to imply the flag (yuck)? A .cmd wrapper script to run python with the flag (ugh)? So you prefer to make the per use site-package directory an opt-in option? I prefer it as an opt-out option. It's enabled by default, unless the user disables the feature with -s. I'm not sure how to solve the problem on Windows. IMHO the feature should be enabled on Windows at least but I like to keep it enabled on all systems. The PEP doesn't add a new attack vector. The problem also exist with PYTHONPATH. Paranoid programs should start with -E -s anyway and paranoid system administrators can switch a flag in site.py: # Enable per user site-packages directory # set it to False to disable the feature or True to force the feature ENABLE_USER_SITE = None If we disable the feature by default it won't be available for a lot of users. For security reasons we also need it disabled when the getuid() != geteuid() to avoid user supplied code being executed as another user. Defaulting to disabled would mean that security could be left up to the end user to mess up. (many systems do not allow setuid #! scripts but this issue would still apply to things run under sudo) It sounds like a reasonable and easy implementable idea, at least on Unix. Windows doesn't have getuid() and geteuid(). On the other hand Windows doesn't have the suid bit, too. I also tried to check if os.stat(__main__.__file__).st_uid == os.getuid() but the real __main__ is not available in site.py. It's loaded and assigned much later. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On 1/13/08, Christian Heimes [EMAIL PROTECTED] wrote: Gregory P. Smith wrote: My main suggestion was going to be the ability to turn it off as you already mentioned. However, please consider leaving it off by default to avoid problems for installed python scripts importing user supplied code. For shared hosting environments where this becomes really useful users can easily add the -s (or whatever flag is chosen) to their programs themselves. I don't know what that'd mean on windows where #! lines don't exist. Yet another file extension to imply the flag (yuck)? A .cmd wrapper script to run python with the flag (ugh)? So you prefer to make the per use site-package directory an opt-in option? I prefer it as an opt-out option. It's enabled by default, unless the user disables the feature with -s. I'm not sure how to solve the problem on Windows. IMHO the feature should be enabled on Windows at least but I like to keep it enabled on all systems. The PEP doesn't add a new attack vector. The problem also exist with PYTHONPATH. Paranoid programs should start with -E -s anyway and paranoid system administrators can switch a flag in site.py: Good point, leave it on by default. # Enable per user site-packages directory # set it to False to disable the feature or True to force the feature ENABLE_USER_SITE = None If we disable the feature by default it won't be available for a lot of users. For security reasons we also need it disabled when the getuid() != geteuid() to avoid user supplied code being executed as another user. Defaulting to disabled would mean that security could be left up to the end user to mess up. (many systems do not allow setuid #! scripts but this issue would still apply to things run under sudo) It sounds like a reasonable and easy implementable idea, at least on Unix. Windows doesn't have getuid() and geteuid(). On the other hand Windows doesn't have the suid bit, too. I also tried to check if os.stat(__main__.__file__).st_uid == os.getuid() but the real __main__ is not available in site.py. It's loaded and assigned much later. Is sys.argv[0] available at that point? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Gregory P. Smith wrote: I also tried to check if os.stat(__main__.__file__).st_uid == os.getuid() but the real __main__ is not available in site.py. It's loaded and assigned much later. Is sys.argv[0] available at that point? No, it's not available, too. The 'site' module is imported by Py_Initialize(). sys.argv and the real __main__ module are set much later in Modules/main.c. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
I've uploaded a new patch: http://bugs.python.org/issue1799 Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Christian Heimes wrote: MA Lemburg has suggested a per user site-packages directory in the pkgutil, pkg_resource and Python 3.0 name space packages thread. I've written a short PEP about it for Python 2.6 and 3.0. Addition: An user has requested a new option to suppress the user site packages directory: -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On 1/12/08, Christian Heimes [EMAIL PROTECTED] wrote: Christian Heimes wrote: MA Lemburg has suggested a per user site-packages directory in the pkgutil, pkg_resource and Python 3.0 name space packages thread. I've written a short PEP about it for Python 2.6 and 3.0. Addition: An user has requested a new option to suppress the user site packages directory: -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE +0.5 Thanks for writing this up as a PEP. My main suggestion was going to be the ability to turn it off as you already mentioned. However, please consider leaving it off by default to avoid problems for installed python scripts importing user supplied code. For shared hosting environments where this becomes really useful users can easily add the -s (or whatever flag is chosen) to their programs themselves. I don't know what that'd mean on windows where #! lines don't exist. Yet another file extension to imply the flag (yuck)? A .cmd wrapper script to run python with the flag (ugh)? For security reasons we also need it disabled when the getuid() != geteuid() to avoid user supplied code being executed as another user. Defaulting to disabled would mean that security could be left up to the end user to mess up. (many systems do not allow setuid #! scripts but this issue would still apply to things run under sudo) -gps ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP: per user site-packages directory
MA Lemburg has suggested a per user site-packages directory in the pkgutil, pkg_resource and Python 3.0 name space packages thread. I've written a short PEP about it for Python 2.6 and 3.0. PEP: XXX Title: Per user site-packages directory Version: $Revision$ Last-Modified: $Date$ Author: Christian Heimes christian(at)cheimes(dot)de Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 11-Jan-2008 Python-Version: 2.6, 3.0 Post-History: Abstract This PEP proposes a new a per user site-packages directory to allow users the local installation of Python packages in their home directory. Rationale = Current Python versions don't have an unified way to install packages into the home directory of an user (except for Mac Framework builds). Users are either forced to ask the system administrator to install or update a package for them or to use one of the many workaround like Virtual Python [1]_, Working Env [2]_ or Virtual Env [3]_. It's not the goal of the PEP to replace the tools or to implement isolated installations of Python. It only implements the most common use case of an additional site-packages directory for each user. The feature can't be implemented using the environment variable *PYTHONPATH*. The env var just inserts a new directory to the beginning of *sys.path* but it doesn't parse the pth files in the directory. A full blown site-packages path is required for several applications and Python eggs. Specification = site directory (site-packages) A directory in sys.path. In contrast to ordinary directories the pth files in the directory are processed, too. user site directory A site directory inside the users' home directory. An user site directory is specific to a Python version. The path contains the version number (major and minor only). Windows: %APPDATA%/Python/Python26/site-packages Mac: ~/Library/Python/2.6/site-packages Unix: ~/.local/lib/python2.6/site-packages user configuration directory Usually the parent directory of the user site directory. It's meant for Python version specific data like config files. Windows: %APPDATA%/Python/Python26 Mac: ~/Library/Python/2.6 Unix: ~/.local/lib/python2.6 user base directory It's located inside the user's home directory. The user site and use config directory are inside the base directory. On some systems the directory may be shared with 3rd party apps. Windows: %APPDATA%/Python Mac: ~/Library/Python Unix: ~/.local On Windows APPDATA was chosen because it is the most logical place for application data. Microsoft recommands that software doesn't write to USERPROFILE [5]_ and My Documents is not suited for application data, too. [8]_ On Linux ~/.local/ was chosen in favor over ./~python/ because the directory is already used by several other programs in analogy to /usr/local. [7]_ Implementation == The site module gets a new method adduserpackage() which adds the appropriate directory to the search path. The directory is not added if it doesn't exist when Python is started. However the location of the user site directory and user base directory is stored in an internal variable for distutils. The user site directory is added before the system site directories but after Python's search paths and PYTHONPATH. This setup allows the user to install a different version of a package than the system administrator but it prevents the user from accidently overwriting a stdlib module. Stdlib modules can still be overwritten with PYTHONPATH. distutils.command.install (setup.py install) gets a new argument --user to install packages in the user site directory. The required directories are created by install. distutils.sysconfig will get methods to access the private variables of site. (not yet implemented) The Windows updater needs to be updated, too. It should create an menu item which opens the user site directory in a new explorer windows. Backwards Compatibility === TBD Open Questions == * Are the directories for Windows, Mac and Unix fine? * Mac: Should framework and non-framework builds of Python use the same directories? * The patch also adds a usecustomize hook to site. Is it useful and should it stay? Reference Implementation A reference implementation is available in the bug tracker. [4]_ Acknowledgments === The implementation of this PEP, if accepted, is sponsored by Google through the Google Summer of Code program. Copyright = This document has been placed in the public domain. References == .. [1] Virtual Python http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python .. [2] Working Env http://pypi.python.org/pypi/workingenv.py http://blog.ianbicking.org/workingenv-revisited.html .. [3] Virtual Env http://pypi.python.org/pypi/virtualenv .. [4] reference implementation
Re: [Python-Dev] PEP: per user site-packages directory
I can't comment on the implementation details, but +1 for the idea. I think this feature will be very useful in a shared hosting environment. -- Alexandre On Jan 11, 2008 6:27 PM, Christian Heimes [EMAIL PROTECTED] wrote: PEP: XXX Title: Per user site-packages directory Version: $Revision$ Last-Modified: $Date$ Author: Christian Heimes christian(at)cheimes(dot)de Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 11-Jan-2008 Python-Version: 2.6, 3.0 Post-History: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
