Re: #! /usr/bin/python{,3} -Es
On Oct 27, 2012, at 06:53 PM, Jakub Wilk wrote: * Thomas Kluyver tho...@kluyver.me.uk, 2012-10-26, 11:03: Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: http://docs.python.org/using/cmdline.html#environment-variables If I set PYTHONWARNINGS, I want it to affect all Python scripts. Not as convenient, but of course you can always run a script with an explicit interpreter command line, instead of using the shebang. Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121028065213.0c00bb59@resist
Re: #! /usr/bin/python{,3} -Es
On 26/10/2012 19:02, Thomas Kluyver wrote: On 26 October 2012 11:53, Chow Loong Jin hyper...@debian.org wrote: What is the definition of system script? Any script installed via dpkg, perhaps? I wouldn't have said so - I install plenty of Python scripts from packages, like /usr/bin/ipython, that I wouldn't call system scripts. I don't think there's a robust distinction on Linux, but there's loosely a difference between system components and user applications. For the purposes of this discussion I would be inclined to define system scripts to be scripts installed by the system administrator, i.e. stuff that gets installed via dpkg, while user scripts are scripts that are installed by the user in his/her $HOME. So with that definition in place, are there any global scripts installed by the system administrator that should honour the PYTHONHOME/PYTHONPATH environment variables? When I run these scripts, I'd expect them to use whatever they came with, or you could end up with $script importing $private_module of a different version and then crashing due to incompatibilities. In the case of django-admin, I'd be inclined to treat it as a global script as described above, as you should be using ./manage.py for your local-repo administration tasks anyway, if you really intend to use it with a virtualenv. And if you really need to use django-admin with a virtualenv, then django-admin.py resolves to the in-virtualenv django-admin. django-admin without the extension is a Debianism. ipython as mentioned in a different post, would be an exception to the rule -- I would want it to the environment variables so I can play with things inside a virtualenv. -- Kind regards, Loong Jin signature.asc Description: OpenPGP digital signature
Re: #! /usr/bin/python{,3} -Es
* Thomas Kluyver tho...@kluyver.me.uk, 2012-10-26, 11:03: Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: http://docs.python.org/using/cmdline.html#environment-variables If I set PYTHONWARNINGS, I want it to affect all Python scripts. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121027165341.ga3...@jwilk.net
Re: #! /usr/bin/python{,3} -Es
On 25 October 2012 20:34, Piotr Ożarowski pi...@debian.org wrote: [Steve Langasek, 2012-10-25] On Thu, Oct 25, 2012 at 10:56:05AM -0400, Barry Warsaw wrote: Using -E fixed the immediate bug, but I think it is generally useful to include -s also, so as to avoid any potential breakage of system scripts by things users may have added locally. If there's consensus that this should be dealt with in the packages, best would be to update the tooling (IIRC dh_python* already have some support for shebang rewrites?) and add a lintian warning, foregoing any mass bug filing. dh_python2 --shebang '/usr/bin/python -Es' dh_python3 --shebang '/usr/bin/python3 -Es' should do the trick. I don't think adding -Es by default is a good idea (although it's very tempting). Several people have said they don't think this is a good idea. But why not? There is a bad effect if you don't rewrite the shebang to /usr/bin/python[3] -ES that we know of, but is there any example of where such a shebang line would cause trouble that warrants not doing this by default? Regards, Floris -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAAWk_Dy4yXr4Xvksw7V6Sc-SYg+3mBZroCqDXnoq1kNF=yu...@mail.gmail.com
Re: #! /usr/bin/python{,3} -Es
On 26 October 2012 09:19, Floris Bruynooghe f...@devork.be wrote: Several people have said they don't think this is a good idea. But why not? There is a bad effect if you don't rewrite the shebang to /usr/bin/python[3] -ES that we know of, but is there any example of where such a shebang line would cause trouble that warrants not doing this by default? Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: http://docs.python.org/using/cmdline.html#environment-variables Thomas -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOvn4qjiW6VicuZnT-CdQnESXkVMc0kO4pVGCV_3vLwx=nd...@mail.gmail.com
Re: #! /usr/bin/python{,3} -Es
* Thomas Kluyver tho...@kluyver.me.uk, 2012-10-26, 11:03: Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: http://docs.python.org/using/cmdline.html#environment-variables What is the definition of system script? -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121026103340.ga3...@jwilk.net
Re: #! /usr/bin/python{,3} -Es
On 26/10/2012 18:33, Jakub Wilk wrote: * Thomas Kluyver tho...@kluyver.me.uk, 2012-10-26, 11:03: Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: http://docs.python.org/using/cmdline.html#environment-variables What is the definition of system script? Any script installed via dpkg, perhaps? -- Kind regards, Loong Jin signature.asc Description: OpenPGP digital signature
Re: #! /usr/bin/python{,3} -Es
On 26 October 2012 11:53, Chow Loong Jin hyper...@debian.org wrote: What is the definition of system script? Any script installed via dpkg, perhaps? I wouldn't have said so - I install plenty of Python scripts from packages, like /usr/bin/ipython, that I wouldn't call system scripts. I don't think there's a robust distinction on Linux, but there's loosely a difference between system components and user applications. Thomas -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOvn4qjjutDhApA1YVBGWY61z9NoLp=r8ctkhggw-jwdnwf...@mail.gmail.com
Re: #! /usr/bin/python{,3} -Es
On Fri, Oct 26, 2012 at 11:03:34AM +0100, Thomas Kluyver wrote: On 26 October 2012 09:19, Floris Bruynooghe f...@devork.be wrote: Several people have said they don't think this is a good idea. But why not? There is a bad effect if you don't rewrite the shebang to /usr/bin/python[3] -ES that we know of, but is there any example of where such a shebang line would cause trouble that warrants not doing this by default? Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: django-admin if I'm using a virtualenv? Perhaps I'm missing the point? Will this just break all virtualenv support? If so, I'm not keen on that. http://docs.python.org/using/cmdline.html#environment-variables Thomas -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOvn4qjiW6VicuZnT-CdQnESXkVMc0kO4pVGCV_3vLwx=nd...@mail.gmail.com -- .''`. Paul Tagliamonte paul...@debian.org : :' : Proud Debian Developer `. `'` 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `- http://people.debian.org/~paultag signature.asc Description: Digital signature
Re: #! /usr/bin/python{,3} -Es
On 26 October 2012 14:20, Paul Tagliamonte paul...@debian.org wrote: django-admin if I'm using a virtualenv? Perhaps I'm missing the point? Will this just break all virtualenv support? To my mind, django-admin is not a system script. A system script would be something like the unattended-upgrades script, which you wouldn't really want to be affected by virtualenvs. Thomas -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caovn4qimsiet2_lwts2p1c37mjabgmi5tsxsy3ptzqgc-vy...@mail.gmail.com
Re: #! /usr/bin/python{,3} -Es
On 26 October 2012 14:20, Paul Tagliamonte paul...@debian.org wrote: On Fri, Oct 26, 2012 at 11:03:34AM +0100, Thomas Kluyver wrote: On 26 October 2012 09:19, Floris Bruynooghe f...@devork.be wrote: Several people have said they don't think this is a good idea. But why not? There is a bad effect if you don't rewrite the shebang to /usr/bin/python[3] -ES that we know of, but is there any example of where such a shebang line would cause trouble that warrants not doing this by default? Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: django-admin if I'm using a virtualenv? Perhaps I'm missing the point? Will this just break all virtualenv support? I don't think a virtualenv would break. If I execute a script which lives outside of the virtualenv I generally want it to work with the python it was installed with. I regularly trip over ipython breaking inside a virtualenv for example. I tend to keep virtualenvs separated from the system python, if I need django-admin in the virtualenv then I should install it there. Regards, Floris -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAAWk_DxR=3teKaEtBU6XyuL=kL4VRtWYs=vgxlngm9xjwya...@mail.gmail.com
Re: #! /usr/bin/python{,3} -Es
On Oct 26, 2012, at 09:19 AM, Floris Bruynooghe wrote: On 25 October 2012 20:34, Piotr Ożarowski pi...@debian.org wrote: [Steve Langasek, 2012-10-25] On Thu, Oct 25, 2012 at 10:56:05AM -0400, Barry Warsaw wrote: Using -E fixed the immediate bug, but I think it is generally useful to include -s also, so as to avoid any potential breakage of system scripts by things users may have added locally. If there's consensus that this should be dealt with in the packages, best would be to update the tooling (IIRC dh_python* already have some support for shebang rewrites?) and add a lintian warning, foregoing any mass bug filing. dh_python2 --shebang '/usr/bin/python -Es' dh_python3 --shebang '/usr/bin/python3 -Es' should do the trick. I don't think adding -Es by default is a good idea (although it's very tempting). Several people have said they don't think this is a good idea. But why not? There is a bad effect if you don't rewrite the shebang to /usr/bin/python[3] -ES that we know of, but is there any example of where such a shebang line would cause trouble that warrants not doing this by default? (Note it's a lower-case 's'. Upper-case 'S' does something different.) I'm also curious as to why this would be generally bad. I can completely understand that there may be some scripts which you want to allow for user customization, but I think those would be the exception, not the rule. That's why I think turning it on by default, with a --disable flag is probably the right way to go. Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121026111518.27c10378@resist
Re: #! /usr/bin/python{,3} -Es
On Oct 26, 2012, at 12:33 PM, Jakub Wilk wrote: * Thomas Kluyver tho...@kluyver.me.uk, 2012-10-26, 11:03: Are there any situations where you might want to run a system script with modified Python environment variables? I can't think of any off the top of my head. Here's the list of environment variables: http://docs.python.org/using/cmdline.html#environment-variables What is the definition of system script? It's a good question. In some sense, it's any script that is vital to the proper functioning of the system, as opposed to applications which run on the system. lsb_release falls under that category for me, but not django-admin. But maybe it's more like that famous line describing obscenity[1], I know it when I see it. Cheers, -Barry [1] http://en.wikipedia.org/wiki/I_know_it_when_I_see_it -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121026112057.67d63f63@resist
Re: #! /usr/bin/python{,3} -Es
* Barry Warsaw ba...@python.org, 2012-10-25, 10:56:
https://bugs.launchpad.net/ubuntu/+source/lsb/+bug/938869
When you write system scripts like 'lsb_release' that use
/usr/bin/python{,3} in the #! line, please remember to include -Es
switches. This prevents the crash in question, where a VMware
installer script was mucking with $PYTHONHOME for its own purposes,
with the side effect of breaking Python's search for its stdlib. The
bug notices it in lsb_release first, and we fixed that in Ubuntu, but
any other Python-based system script that the VMware script would have
called would also be broken.
Surely that's a bug in VMware, not in lsb_release?
--
Jakub Wilk
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20121025170508.ga3...@jwilk.net
Re: #! /usr/bin/python{,3} -Es
On Oct 25, 2012, at 07:05 PM, Jakub Wilk wrote:
* Barry Warsaw ba...@python.org, 2012-10-25, 10:56:
https://bugs.launchpad.net/ubuntu/+source/lsb/+bug/938869
When you write system scripts like 'lsb_release' that use
/usr/bin/python{,3} in the #! line, please remember to include -Es
switches. This prevents the crash in question, where a VMware installer
script was mucking with $PYTHONHOME for its own purposes, with the side
effect of breaking Python's search for its stdlib. The bug notices it in
lsb_release first, and we fixed that in Ubuntu, but any other Python-based
system script that the VMware script would have called would also be
broken.
Surely that's a bug in VMware, not in lsb_release?
IIUC, VMware is doing this because they're shipping their own Python stdlib
and want their Python tools (invoked by the script) to use that. So yeah, I'd
agree that VMware shouldn't be doing that. ;)
It still makes sense to insulate our system scripts from such silliness, and
adding -Es should become second nature. But I also don't think it's a
widespread problem in practice, which is why I advocated against a mass
freakout. :)
Cheers,
-Barry
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/2012102514.374c9...@limelight.wooz.org
Re: #! /usr/bin/python{,3} -Es
On Thu, Oct 25, 2012 at 10:56:05AM -0400, Barry Warsaw wrote: This doesn't requires a mass freakout, but it might be useful for a mass bug filing (of non-urgent priority, I think). I don't have time before UDS-R to look into that, so I at least wanted to send this email to put it on the radar. Here is what the switches do (from python -h): -E : ignore PYTHON* environment variables (such as PYTHONPATH) -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE Using -E fixed the immediate bug, but I think it is generally useful to include -s also, so as to avoid any potential breakage of system scripts by things users may have added locally. If there's consensus that this should be dealt with in the packages, best would be to update the tooling (IIRC dh_python* already have some support for shebang rewrites?) and add a lintian warning, foregoing any mass bug filing. But like Jakub I'm not sure this actually warrants proactive effort on our part, because the only instance of this we've seen so far can be attributed to a misbehaving third-party app tainting the environment. Yes, it's reasonable to work around that one known case, but why spend any effort on this problem unless and until we see a pattern of such abuse (where a pattern is N1)? -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Re: #! /usr/bin/python{,3} -Es
On Oct 25, 2012, at 11:18 AM, Steve Langasek wrote: On Thu, Oct 25, 2012 at 10:56:05AM -0400, Barry Warsaw wrote: This doesn't requires a mass freakout, but it might be useful for a mass bug filing (of non-urgent priority, I think). I don't have time before UDS-R to look into that, so I at least wanted to send this email to put it on the radar. Here is what the switches do (from python -h): -E : ignore PYTHON* environment variables (such as PYTHONPATH) -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE Using -E fixed the immediate bug, but I think it is generally useful to include -s also, so as to avoid any potential breakage of system scripts by things users may have added locally. If there's consensus that this should be dealt with in the packages, best would be to update the tooling (IIRC dh_python* already have some support for shebang rewrites?) and add a lintian warning, foregoing any mass bug filing. But like Jakub I'm not sure this actually warrants proactive effort on our part, because the only instance of this we've seen so far can be attributed to a misbehaving third-party app tainting the environment. Yes, it's reasonable to work around that one known case, but why spend any effort on this problem unless and until we see a pattern of such abuse (where a pattern is N1)? I do believe dh_python* rewrites shebang lines already, so adding this there is probably both easy, and appropriate (with a --disable flag of course). I agree it's not worth worrying about unless we see a pattern of breakage. (OTOH, if it's easy enough to add and you happen to be mucking about in this area already... :). -Barry signature.asc Description: PGP signature
Re: #! /usr/bin/python{,3} -Es
[Steve Langasek, 2012-10-25] On Thu, Oct 25, 2012 at 10:56:05AM -0400, Barry Warsaw wrote: Using -E fixed the immediate bug, but I think it is generally useful to include -s also, so as to avoid any potential breakage of system scripts by things users may have added locally. If there's consensus that this should be dealt with in the packages, best would be to update the tooling (IIRC dh_python* already have some support for shebang rewrites?) and add a lintian warning, foregoing any mass bug filing. dh_python2 --shebang '/usr/bin/python -Es' dh_python3 --shebang '/usr/bin/python3 -Es' should do the trick. I don't think adding -Es by default is a good idea (although it's very tempting). I can add --ignore-custom-site-dirs-in-scripts option to extend existing args in shebangs if you think such option would help. -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121025193418.gc8...@piotro.eu
