Re: [Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values
On 2016-03-02 18:45, Thomas Wouters wrote: On Wed, Mar 2, 2016 at 3:50 AM, Michael Feltwrote: Hello all, 1) There are many lists to choose from - if this is the wrong one for questions about packaging - please forgive me, and point me in the right direction. It's hard to say where this belongs best, but python-list would probably have done as well. 2) Normally, I have just packaged python, and then moved on. However, recently I have been asked to help with packaging an 'easier to install' python by people using cloud-init, and more recently people wanting to use salt-stack (on AIX). FYI: I have been posting about my complete failure to build 2.7.11 ( http://bugs.python.org/issue26466) - so, what I am testing is based on 2.7.10 - which built easily for me. Going through the 'base documentation' I saw a reference to both sys.argv and sys.path. atm, I am looking for an easy way to get the program name (e.g., /opt/bin/python, versus ./python). I have my reasons (basically, looking for a compiled-in library search path to help with http://bugs.python.org/issue26439) I think the only way to get at the compiled-in search path is to recreate it based on the compiled-in prefix, which you can get through distutils. Python purposely only uses the compiled-in path as the last resort. Instead, it searches for its home relative to the executable and adds a set of directories relative to its home (if they exist). It's not clear to me why you're focusing on these differences, as (as I describe below) they are immaterial. Looking on two platforms (AIX, my build, and debian for power) I am surprised that sys.argv is empty in both cases, and sys.path returns /opt/lib/python27.zip with AIX, but not with debian. When you run python interactively, sys.argv[0] will be '', yes. Since you're not launching a program, there's nothing else to set it to. 'python' (or the path to the executable) wouldn't be the right thing to set it to, because python itself isn't a Python program :) The actual python executable is sys.executable, not sys.argv[0], but you shouldn't usually care about that, either. If you want to know where to install things, distutils is the thing to use. If you want to know where Python thinks it's installed (for debugging purposes only, really), sys.prefix will tell you. root@x064:[/data/prj/aixtools/python/python-2.7.10]/opt/bin/python Python 2.7.10 (default, Nov 3 2015, 14:36:51) [C] on aix5 Type "help", "copyright", "credits" or "license" for more information. import sys sys.argv [''] sys.path ['', '/opt/lib/python27.zip', '/opt/lib/python2.7', '/opt/lib/python2.7/plat-aix5', '/opt/lib/python2.7/lib-tk', '/opt/lib/python2.7/lib-old', '/opt/lib/python2.7/lib-dynload', '/opt/lib/python2.7/site-packages'] michael@ipv4:~$ python Python 2.7.9 (default, Mar 1 2015, 13:01:00) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. import sys sys.argv [''] sys.path ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-powerpc-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7'] In sys.path, you're seeing the difference between a vanilla Python and Debian's patched Python. Vanilla Python adds $prefix/lib/python27.zip to sys.path unconditionally, whereas Debian removes it when it doesn't exist. Likewise, the dist-packages directory is a local modification by Debian; in vanilla Python it's called 'site-packages' instead. The subdirectories in dist-packages that you see in the Debian case are added by .pth files installed in $prefix -- third-party packages, in other words, adding their own directories to the module search path. And I guess I would be interested in getting '/opt/lib/python2.7/dist-packages' in there as well (or learn a way to later add it for pre-compiled packages such as cloud-init AND that those would also look 'first' in /opt/lib/python2.7/dist-packages/cloud-init for modules added to support cloud-init - should I so choose (mainly in case of compatibility issues between say cloud-init and salt-stack that have common modules BUT may have conflicts) - Hopefully never needed for that reason, but it might also simplify packaging applications that depend on python. A vanilla Python (or non-Debian-built python, even) has no business looking in dist-packages. It should just use site-packages. Third-party packages shouldn't care whether they're installed in site-packages or dist-packages, and instead should use distutils one way or another (if not by having an actual setup.py that uses distutils or setuptools, then at least by querying distutils for the installation directory the way python-config does). Many thanks for your time and pointers into the documentation,
Re: [Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values
On Wed, Mar 2, 2016 at 3:50 AM, Michael Feltwrote: > Hello all, > > 1) There are many lists to choose from - if this is the wrong one for > questions about packaging - please forgive me, and point me in the right > direction. > It's hard to say where this belongs best, but python-list would probably have done as well. > > 2) Normally, I have just packaged python, and then moved on. However, > recently I have been asked to help with packaging an 'easier to install' > python by people using cloud-init, and more recently people wanting to use > salt-stack (on AIX). > > FYI: I have been posting about my complete failure to build 2.7.11 ( > http://bugs.python.org/issue26466) - so, what I am testing is based on > 2.7.10 - which built easily for me. > > Going through the 'base documentation' I saw a reference to both sys.argv > and sys.path. atm, I am looking for an easy way to get the program name > (e.g., /opt/bin/python, versus ./python). > I have my reasons (basically, looking for a compiled-in library search > path to help with http://bugs.python.org/issue26439) > I think the only way to get at the compiled-in search path is to recreate it based on the compiled-in prefix, which you can get through distutils. Python purposely only uses the compiled-in path as the last resort. Instead, it searches for its home relative to the executable and adds a set of directories relative to its home (if they exist). It's not clear to me why you're focusing on these differences, as (as I describe below) they are immaterial. > Looking on two platforms (AIX, my build, and debian for power) I am > surprised that sys.argv is empty in both cases, and sys.path returns > /opt/lib/python27.zip with AIX, but not with debian. > When you run python interactively, sys.argv[0] will be '', yes. Since you're not launching a program, there's nothing else to set it to. 'python' (or the path to the executable) wouldn't be the right thing to set it to, because python itself isn't a Python program :) The actual python executable is sys.executable, not sys.argv[0], but you shouldn't usually care about that, either. If you want to know where to install things, distutils is the thing to use. If you want to know where Python thinks it's installed (for debugging purposes only, really), sys.prefix will tell you. > > root@x064:[/data/prj/aixtools/python/python-2.7.10]/opt/bin/python > Python 2.7.10 (default, Nov 3 2015, 14:36:51) [C] on aix5 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> sys.argv > [''] > >>> sys.path > ['', '/opt/lib/python27.zip', '/opt/lib/python2.7', > '/opt/lib/python2.7/plat-aix5', '/opt/lib/python2.7/lib-tk', > '/opt/lib/python2.7/lib-old', '/opt/lib/python2.7/lib-dynload', > '/opt/lib/python2.7/site-packages'] > > michael@ipv4:~$ python > Python 2.7.9 (default, Mar 1 2015, 13:01:00) > [GCC 4.9.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> sys.argv > [''] > >>> sys.path > ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-powerpc-linux-gnu', > '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', > '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', > '/usr/lib/python2.7/dist-packages', > '/usr/lib/python2.7/dist-packages/PILcompat', > '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7'] > In sys.path, you're seeing the difference between a vanilla Python and Debian's patched Python. Vanilla Python adds $prefix/lib/python27.zip to sys.path unconditionally, whereas Debian removes it when it doesn't exist. Likewise, the dist-packages directory is a local modification by Debian; in vanilla Python it's called 'site-packages' instead. The subdirectories in dist-packages that you see in the Debian case are added by .pth files installed in $prefix -- third-party packages, in other words, adding their own directories to the module search path. > > And I guess I would be interested in getting > '/opt/lib/python2.7/dist-packages' in there as well (or learn a way to > later add it for pre-compiled packages such as cloud-init AND that those > would also look 'first' in /opt/lib/python2.7/dist-packages/cloud-init for > modules added to support cloud-init - should I so choose (mainly in case of > compatibility issues between say cloud-init and salt-stack that have common > modules BUT may have conflicts) - Hopefully never needed for that reason, > but it might also simplify packaging applications that depend on python. > A vanilla Python (or non-Debian-built python, even) has no business looking in dist-packages. It should just use site-packages. Third-party packages shouldn't care whether they're installed in site-packages or dist-packages, and instead should use distutils one way or another (if not by having an actual setup.py that uses distutils or setuptools, then at least by querying distutils for the installation directory the way
Re: [Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values
On Wed, 2 Mar 2016 at 09:12 Michael Feltwrote: > Hello all, > > 1) There are many lists to choose from - if this is the wrong one for > questions about packaging - please forgive me, and point me in the right > direction. > So in this instance you're after python-list since this is a general support question. But since I have an answer for you... > > 2) Normally, I have just packaged python, and then moved on. However, > recently I have been asked to help with packaging an 'easier to install' > python by people using cloud-init, and more recently people wanting to > use salt-stack (on AIX). > > FYI: I have been posting about my complete failure to build 2.7.11 ( > http://bugs.python.org/issue26466) - so, what I am testing is based on > 2.7.10 - which built easily for me. > > Going through the 'base documentation' I saw a reference to both > sys.argv and sys.path. atm, I am looking for an easy way to get the > program name (e.g., /opt/bin/python, versus ./python). > I have my reasons (basically, looking for a compiled-in library search > path to help with http://bugs.python.org/issue26439) > https://docs.python.org/2.7/library/sys.html#sys.executable > > Looking on two platforms (AIX, my build, and debian for power) I am > surprised that sys.argv is empty in both cases, and sys.path returns > /opt/lib/python27.zip with AIX, but not with debian. > Did you actually build your version of Python on Debian? If not then do realize that Debian patches their version of CPython, so it wouldn't shock me if they stripped out the code that adds the zip file to sys.path. IOW don't trust the pre-installed CPython to act the same as one that is built from source. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values
Hello all, 1) There are many lists to choose from - if this is the wrong one for questions about packaging - please forgive me, and point me in the right direction. 2) Normally, I have just packaged python, and then moved on. However, recently I have been asked to help with packaging an 'easier to install' python by people using cloud-init, and more recently people wanting to use salt-stack (on AIX). FYI: I have been posting about my complete failure to build 2.7.11 ( http://bugs.python.org/issue26466) - so, what I am testing is based on 2.7.10 - which built easily for me. Going through the 'base documentation' I saw a reference to both sys.argv and sys.path. atm, I am looking for an easy way to get the program name (e.g., /opt/bin/python, versus ./python). I have my reasons (basically, looking for a compiled-in library search path to help with http://bugs.python.org/issue26439) Looking on two platforms (AIX, my build, and debian for power) I am surprised that sys.argv is empty in both cases, and sys.path returns /opt/lib/python27.zip with AIX, but not with debian. root@x064:[/data/prj/aixtools/python/python-2.7.10]/opt/bin/python Python 2.7.10 (default, Nov 3 2015, 14:36:51) [C] on aix5 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.argv [''] >>> sys.path ['', '/opt/lib/python27.zip', '/opt/lib/python2.7', '/opt/lib/python2.7/plat-aix5', '/opt/lib/python2.7/lib-tk', '/opt/lib/python2.7/lib-old', '/opt/lib/python2.7/lib-dynload', '/opt/lib/python2.7/site-packages'] michael@ipv4:~$ python Python 2.7.9 (default, Mar 1 2015, 13:01:00) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.argv [''] >>> sys.path ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-powerpc-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7'] And I guess I would be interested in getting '/opt/lib/python2.7/dist-packages' in there as well (or learn a way to later add it for pre-compiled packages such as cloud-init AND that those would also look 'first' in /opt/lib/python2.7/dist-packages/cloud-init for modules added to support cloud-init - should I so choose (mainly in case of compatibility issues between say cloud-init and salt-stack that have common modules BUT may have conflicts) - Hopefully never needed for that reason, but it might also simplify packaging applications that depend on python. Many thanks for your time and pointers into the documentation, It is a bit daunting :) Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com