[Python-Dev] Re: Why doesn't venv also install python3*-config?
On 13.01.2020 16:21, Daniel Haude wrote: Am 12.01.2020 23:39 schrieb Ivan Pozdeev via Python-Dev: Virtualenv does create `python-config` shim in Linux. Python3 doesn't. Definetely. Not on RHEL7 and Debian, that's where I tested it. You are right, I've got it mixed up. Virtualenv creates `python-config`. Venv doesn't. $ pyenv local 3.6.9 $ pip install virtualenv <...> $ virtualenv test_vi Using base prefix '/home/vmuser/.pyenv/versions/3.6.9' New python executable in /home/vmuser/test_vi/bin/python3.6 Also creating executable in /home/vmuser/test_vi/bin/python Installing setuptools, pip, wheel... done. $ python -m venv test_ve $ ls test_ve/bin/ activate activate.csh activate.fish easy_install easy_install-3.6 pip pip3 pip3.6 python python3 $ ls test_vi/bin/ activate activate.fish activate_this.py easy_install pip pip3.6 python3 python-config activate.csh activate.ps1 activate.xsh easy_install-3.6 pip3 python python3.6 wheel -- Regards, Ivan ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JPSSRLY5HLVRG6FMX53ZU2XIXB6LY5XI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On 1/13/2020 8:21 AM, Daniel Haude wrote: Am 12.01.2020 23:39 schrieb Ivan Pozdeev via Python-Dev: Virtualenv does create `python-config` shim in Linux. Python3 doesn't. Definetely. Not on RHEL7 and Debian, that's where I tested it. Just to ensure there's no confusion; note that "venv" and "virtualenv" are two related but different packages (venv being included in the standard library and virtualenv available on pypi). At least here on Ubuntu 18.04 with python 3.6 venv and virtualenv 16.7.9, venv does not include python-config and virtualenv does. Janzert ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UT4F4JEADTZX5IH5QMZW66A4WJIOS2KV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
Am 12.01.2020 23:39 schrieb Ivan Pozdeev via Python-Dev: Virtualenv does create `python-config` shim in Linux. Python3 doesn't. Definetely. Not on RHEL7 and Debian, that's where I tested it. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/52ZEDRORRPB4NRW54WIRVSCKY34RUM2Y/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
> On 12 Jan 2020, at 16:20, mus...@posteo.org wrote:
>
> Hi guys,
>
> after I got the whole list into a lather about the merits of
> the python-config program, let me rephrase the question:
>
> Is there a "canonical" way of automatically finding the correct
> include files and Python runtime library when embedding the Python
> interpreter, based on the current virtual environment (and not the
> "installed" version)? Upon reding the docs there isn't. And maybe there
> should't or cant be any. So if I want to try embedding Python under
> different versions I have to either install those into different
> directories and use python-config, or I write a trivial Python program
> that finds the correct values for the current environment using
> sysconfig and outpus those as compiler / linker flags. Easy enough, I
> was just surprised that no such solution was already built into the
> virtualenv setup mechanism
>
> Here's a quote from the docs
> """
> If this procedure [using python-config] doesn’t work for you (it is not
> guaranteed to work for all Unix-like platforms; however, we welcome bug
> reports) you will have to read your system’s documentation about
> dynamic linking and/or examine Python’s Makefile (use
> sysconfig.get_makefile_filename() to find its location) and compilation
> options. In this case, the sysconfig module is a useful tool to
> programmatically extract the configuration values that you will want to
> combine together. For example:
>
import sysconfig
sysconfig.get_config_var('LIBS')
> '-lpthread -ldl -lutil'
sysconfig.get_config_var('LINKFORSHARED')
> '-Xlinker -export-dynamic'
> """
I build a python C++ extension called pysvn. It's build system works
by running a python program to generate a native makefile for
Windows, macOS, Fedora, NetBSD etc. The code support
python 2 and all the python 3's.
I have all the python versions installed on my build systems
side-by-side. I did not need to use venv as all the python's
are installled into the systems.
Maybe you can use some of the ideas from the code.
See this module:
https://svn.code.sf.net/p/pysvn/code/trunk/pysvn/Extension/Source/setup_configure.py
Barry
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/364Q6EZHSFN6DGUUDN3FKOMK4CDS3WJB/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/MIGP4B6FD4HFAHK4OT5SSGQDHNR74TB6/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On 12.01.2020 19:20, mus...@posteo.org wrote:
Hi guys,
after I got the whole list into a lather about the merits of
the python-config program, let me rephrase the question:
Is there a "canonical" way of automatically finding the correct
include files and Python runtime library when embedding the Python
interpreter, based on the current virtual environment (and not the
"installed" version)? Upon reding the docs there isn't.
+ '-config'
Is what
https://docs.python.org/3/extending/embedding.html#compiling-and-linking-under-unix-like-systems
seems to be suggesting to me.
This is not applicable to Windows
And maybe there
should't or cant be any. So if I want to try embedding Python under
different versions I have to either install those into different
directories and use python-config, or I write a trivial Python program
that finds the correct values for the current environment using
sysconfig and outpus those as compiler / linker flags. Easy enough, I
was just surprised that no such solution was already built into the
virtualenv setup mechanism
Virtualenv does create `python-config` shim in Linux.
Here's a quote from the docs
"""
If this procedure [using python-config] doesn’t work for you (it is not
guaranteed to work for all Unix-like platforms; however, we welcome bug
reports) you will have to read your system’s documentation about
dynamic linking and/or examine Python’s Makefile (use
sysconfig.get_makefile_filename() to find its location) and compilation
options. In this case, the sysconfig module is a useful tool to
programmatically extract the configuration values that you will want to
combine together. For example:
import sysconfig
sysconfig.get_config_var('LIBS')
'-lpthread -ldl -lutil'
sysconfig.get_config_var('LINKFORSHARED')
'-Xlinker -export-dynamic'
"""
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/364Q6EZHSFN6DGUUDN3FKOMK4CDS3WJB/
Code of Conduct: http://python.org/psf/codeofconduct/
--
Regards,
Ivan
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/EZ56AYUT52Y7PBV2ORHMFC7M4FNFWNE5/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
Hi guys,
after I got the whole list into a lather about the merits of
the python-config program, let me rephrase the question:
Is there a "canonical" way of automatically finding the correct
include files and Python runtime library when embedding the Python
interpreter, based on the current virtual environment (and not the
"installed" version)? Upon reding the docs there isn't. And maybe there
should't or cant be any. So if I want to try embedding Python under
different versions I have to either install those into different
directories and use python-config, or I write a trivial Python program
that finds the correct values for the current environment using
sysconfig and outpus those as compiler / linker flags. Easy enough, I
was just surprised that no such solution was already built into the
virtualenv setup mechanism
Here's a quote from the docs
"""
If this procedure [using python-config] doesn’t work for you (it is not
guaranteed to work for all Unix-like platforms; however, we welcome bug
reports) you will have to read your system’s documentation about
dynamic linking and/or examine Python’s Makefile (use
sysconfig.get_makefile_filename() to find its location) and compilation
options. In this case, the sysconfig module is a useful tool to
programmatically extract the configuration values that you will want to
combine together. For example:
>>> import sysconfig
>>> sysconfig.get_config_var('LIBS')
'-lpthread -ldl -lutil'
>>> sysconfig.get_config_var('LINKFORSHARED')
'-Xlinker -export-dynamic'
"""
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/364Q6EZHSFN6DGUUDN3FKOMK4CDS3WJB/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On Thu, 9 Jan 2020 15:18:55 +0100 Victor Stinner wrote: > Which build command rely on python-config? Is it to cross-compile 3rd > party C extensions on the host? Hello, OP speaking here. In two cases I came across this issue. No cross-compiling involved. 1) To debug a Python C extension I wrote a standalone C test program which of course needs to find "Python.h" and libpython*.* for building. I ran into trouble when I tried to build that program in a custom compiled debugging Python environment. 2) I need to recheck this, but it was a "pip install" for a package from pypi org. Probably numpy or pandas errored out with not being able to find "Python.h". I'll try and re-check. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KXHEX6UGOMPWOWQPGD7YQTAWECR4Q3KS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
Le jeu. 9 janv. 2020 à 17:19, Xavier de Gaye a écrit : > A cross-compilation means that there is probably no build framework on the > target platform and therefore the build configuration of the > cross-compilation of Python is not very useful there. Ok, so the shell script python-config is useful cross-compile things for Python on the host. But it doesn't go against the idea of providing the same feature as "python3 -m xxx ...": both can coexist ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MKSN5BE7C3B35ZBAGIOA6LL4RHHD2MKE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On 1/9/20 3:18 PM, Victor Stinner wrote: Is the "ARM64" python-config (shell script) executed on the x86-64 host? A cross-compilation means that there is probably no build framework on the target platform and therefore the build configuration of the cross-compilation of Python is not very useful there. Which build command rely on python-config? Is it to cross-compile 3rd party C extensions on the host? It is useful when cross-compiling an application that embeds Python. And I have a hunch that this may be also useful to fix the cross-compilation of third-party extension modules :) Xavier ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2DRE2EEMJ4IYNC52ZV6HA4FNNIK5PRL4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
Le jeu. 9 janv. 2020 à 12:11, Xavier de Gaye a écrit : > The shell script python-config has been introduced by bpo issue 16235 named > "Add python-config.sh for use during cross compilation" in order "to behave > exactly the same as python-config.py except it doesn't depend on a Python > interpreter" as stated in the OP. So it does not make sense to say that one > may pick the wrong python-config script. Its purpose is to provide > cross-compilation configuration data independently of any Python interpreter > running natively on the build machine used to do the cross-compilation. I'm thinking aloud to try to understand how it's supposed to work. Let's say that I cross-compile Python on x86-64 targeting ARM64. The build creates a ARM64 "python" program which cannot be executed on the x86-64 host. A python-config script which contains the target configuration is generated. Is the "ARM64" python-config (shell script) executed on the x86-64 host? Which build command rely on python-config? Is it to cross-compile 3rd party C extensions on the host? Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZTYARNCVQW4EBRVNIZ32VAUBMULSL77D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On 1/8/20 5:53 PM, Victor Stinner wrote: You may get the wrong information if you pick the wrong python-config script :-( IMHO we should add a new module (problem: how should it be called? pyconfig?) The shell script python-config has been introduced by bpo issue 16235 named "Add python-config.sh for use during cross compilation" in order "to behave exactly the same as python-config.py except it doesn't depend on a Python interpreter" as stated in the OP. So it does not make sense to say that one may pick the wrong python-config script. Its purpose is to provide cross-compilation configuration data independently of any Python interpreter running natively on the build machine used to do the cross-compilation. FWIW the shell script python-config is generated from Misc/python-config.sh by configure and updated by the Makefile. Xavier ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UNIJPEYGZHDW2UGBK4JNI5TEDY3MDDPF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On 2020-01-08 17:53, Victor Stinner wrote: Hi, I dislike python-config for multiple reasons You may get the wrong information if you pick the wrong python-config script :-( IMHO we should add a new module (problem: how should it be called? pyconfig?) which could be used using "python3 -m xxx ...". There is a similar discussion between "pip ..." and "python3 -m pip ..." commands: I understand that "python3 -m pip ..." is more reliable to know which Python will be picked, especially in a virtual environment. Indeed. It's a bad idea to have separate executables that use/affect "the Python" or are needed to work with "the Python", but don't precisely specify "the Python". And `python -m ...` is the most sane way out of the mess. There are two implementations: one in Python used on macOS, one in shell used on other platforms. Moreover, if it's implemented in the stdlib, it can be implemented in Python and it should be easier to maintain than a shell script. For example, python-config has no unit test... Could sysconfig be extended? Currently it doesn't respond to CLI flags; it should be possible to add config-like ones, e.g.: python -m sysconfig --libs python -m sysconfig --help Would that help? I must admit I don't really understand python-config. Does anyone know its purpose? History? Documentation? One thing I suspect is that it was meant as a helper for pkg-config, not to be used alone. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FTKE7GRT4IWHJOBX5JPZQXTR42ZT6ZH6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On Wed, Jan 08, 2020 at 12:26:39PM +0100, Musbur wrote: > I'm experimenting with package development on different versions of Python > in different virtualenvs. After running "make" I don't do "make install", > but rather I set up virtualenvs by running /path/to/source/python -m venv > env_dir. I'd suggest https://github.com/pyenv/pyenv -- this handles this more seamlessly. m -- Matt Billenstein m...@vazor.com http://www.vazor.com/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DL45OWIVYPZLKMLVSYISPG2XNY2S3PE7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
On 08.01.2020 14:26, Musbur wrote: Hello, I'm experimenting with package development on different versions of Python in different virtualenvs. After running "make" I don't do "make install", but rather I set up virtualenvs by running /path/to/source/python -m venv env_dir. This works for as long as I don't need to compile extensions. Once I do that I'm running into trouble because there is no python3-config binary in the venv, so it uses the "system" python3-config which of course returns results for the /usr(/local)/ tree. This seems to go against the idea of an encapsulated and self-contained environment. And the venv created straight from the "not-installed" source tree works so well that having a venv/bin/python3-config whose output points into the source tree seems a logical step. Is this an omission or is there a rationale for not doing it? AFAIK for _inside_ a virtual environment, the convention is `python', `python-`, `pip' etc. -- unversioned executables. Versioned executables are the convention for UNIX _outside_ a virtual environment. This is codified in https://www.python.org/dev/peps/pep-0394/. Of course I can "properly" install different Python versions by using different "configure --prefix" directories. But I like the venv so much in general that this rubs me the wrong way. Thanks! ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KMG3USMQLQNI6AEX6UUS2WTNFIIIS2XY/ Code of Conduct: http://python.org/psf/codeofconduct/ -- Regards, Ivan ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N3E26CJGWBI4IJ4VWNXIRAG3CCVYCFPF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Why doesn't venv also install python3*-config?
Hi, I dislike python-config for multiple reasons You may get the wrong information if you pick the wrong python-config script :-( IMHO we should add a new module (problem: how should it be called? pyconfig?) which could be used using "python3 -m xxx ...". There is a similar discussion between "pip ..." and "python3 -m pip ..." commands: I understand that "python3 -m pip ..." is more reliable to know which Python will be picked, especially in a virtual environment. There are two implementations: one in Python used on macOS, one in shell used on other platforms. Moreover, if it's implemented in the stdlib, it can be implemented in Python and it should be easier to maintain than a shell script. For example, python-config has no unit test... Victor Le mer. 8 janv. 2020 à 17:00, Musbur a écrit : > > Hello, > > I'm experimenting with package development on different versions of > Python in different virtualenvs. After running "make" I don't do "make > install", but rather I set up virtualenvs by running > /path/to/source/python -m venv env_dir. This works for as long as I > don't need to compile extensions. Once I do that I'm running into > trouble because there is no python3-config binary in the venv, so it > uses the "system" python3-config which of course returns results for the > /usr(/local)/ tree. > > This seems to go against the idea of an encapsulated and self-contained > environment. And the venv created straight from the "not-installed" > source tree works so well that having a venv/bin/python3-config whose > output points into the source tree seems a logical step. Is this an > omission or is there a rationale for not doing it? > > Of course I can "properly" install different Python versions by using > different "configure --prefix" directories. But I like the venv so much > in general that this rubs me the wrong way. > > Thanks! > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/KMG3USMQLQNI6AEX6UUS2WTNFIIIS2XY/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NTNC4ZXNR6XFJPX5HK6VEFCZYEPP47IV/ Code of Conduct: http://python.org/psf/codeofconduct/
