[issue18309] Make python slightly more relocatable

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue18309] Make python slightly more relocatable

Shakeeb Alireza added the comment: Thanks, Mathias. This is all about improving python's 'relocatability'. Just to expand on my prior point: the scenario we are talking about is where one embeds python in a host application's plugin system rather than in the host application itself. In

[issue18309] Make python slightly more relocatable

Mathias Fröhlich added the comment: Hey, Shakeeb Alireza is right, the original problem was an application that links and embeds against libpython*{so,dll,dynlib} and should more easily find components like /*/lib*/python*.*/site.py and most probably now it needs to find getpath.py as well.

[issue18309] Make python slightly more relocatable

Shakeeb Alireza added the comment: Thanks, Victor. I can imagine getpath.py will be more hackable (even if it is frozen). Still, it replicates the old algorithm: # Before any searches are done, the location of the executable is # determined. If Py_SetPath() was called, or if we are

[issue18309] Make python slightly more relocatable

STINNER Victor added the comment: In Python 3.11, Modules/getpath.c has been rewritten in Python: Modules/getpath.py. Maybe it's now simpler to hack this file. But you must rebuild Python to take changes in account. -- ___ Python tracker

[issue18309] Make python slightly more relocatable

Shakeeb Alireza added the comment: I have exactly the same need and use-case as Mathias in my project which includes a requirement to embed python3 in a relocatable folder structure w which serves as an application package (https://github.com/shakfu/py-js). This can be done using the

[issue18309] Make python slightly more relocatable

Mathias Fröhlich added the comment: Ok, so far. But what shall I do now? It would be nice that python is a bit smarter in finding its increasing important module files when being embedded into an application. Anybody out there who wants to look at that contribution? best Mathias --

[issue18309] Make python slightly more relocatable

STINNER Victor added the comment: My plan is not to change the default implementation to calculate the path configuration, but make it easier to customize the path configuration. One idea is to rewrite Modules/getpath.c and PC/getpathp.c in Python and convert it to a frozen module. It is

[issue18309] Make python slightly more relocatable

Mathias Fröhlich added the comment: Yes. msg191944 from Nick Coghlan, made me think that with all the initialization rework that appeared to be underway you want to incorporate that presented idea of basing the default onto the location of the libpython.so or the pythonX.X.dll instead of

[issue18309] Make python slightly more relocatable

STINNER Victor added the comment: Hum, I am confused. I understood that this issue is able customizing sys.path when Python is embedded. But it seems like the feature request is more about the *default* implementation, not how to reimplement it outside Python (with custm code). --

[issue18309] Make python slightly more relocatable

Mathias Fröhlich added the comment: Hi, Nice to see some progress. Still, I checked todays https://github.com/python/cpython.git master and 3.8 branch (is that the current cpython development code?). Neither of them contain a call to dladdr beside the macos code path mentioned in msg191994

[issue18309] Make python slightly more relocatable

STINNER Victor added the comment: The PEP 587 "Python Initialization Configuration" has been implemented in Python 3.8. It provides fine control on the "Path Configuration": * https://docs.python.org/dev/c-api/init_config.html *

[issue18309] Make python slightly more relocatable

Mathias Fröhlich added the comment: Hi Ronald, Eric, Nick, Looking up the symbol name of the current function should work also. And I am free to rename these functions to whatever you like. Attached is version 2 of the patch with the suggested changes. The windows implementation is still

[issue18309] Make python slightly more relocatable

Ronald Oussoren added the comment: Note that the OSX port already does this for framework builds. I don't know why we don't use the same code for shared library builds. Issue #15498 contains a patch that switches this code from a deprecated nextstep-era API to dladdr. Two comments on the

[issue18309] Make python slightly more relocatable

Mathias Fröhlich added the comment: Hi Eric, Thanks for looking at that ticket so fast! Reassigning this to 3.4 is great. In general, yes I can already do what I need more or less. This is the reason why I can be fine with about every python version. The point I bring up this change that I

[issue18309] Make python slightly more relocatable

Nick Coghlan added the comment: The way we figure out where to find the standard library is crazy, and creating the infrastructure to start making it less crazy is actually one of the prime motivations for PEP 432 :) -- ___ Python tracker

[issue18309] Make python slightly more relocatable

New submission from Mathias Fröhlich: Hi all, I want to move python a bit closer to be relocatable. One problem to solve is where python finds its modules. The usual lookup mechanism is to compile in a configure time determined prefix that is used as a last resort path if the paths are not set

[issue18309] Make python slightly more relocatable

Eric Snow added the comment: Hi Mathias. There is a current proposal (http://www.python.org/dev/peps/pep-0432/) for improving interpreter startup. So changes in this area are subject to extra caution. The changes you are talking about are at least indirectly impacted by the proposal,