On Mon, 20 Apr 2020 20:29:05 +0200 Davide Andreoli <[email protected]> said:
> your first idea was in the right direction ! > > if I change e_module_new to: > dlopen(modpath, (RTLD_NOW | RTLD_GLOBAL)); > edgar works again as expected \o/ ooh i just removed local.. didn't explicitly add global > I think this make sense because the edgar module.so is linked to python.so, > but the python-efl bindings modules are not. Thus importing python modules > require that the caller (the python interpreter or any app embedding > python) > must be linked to python.so. At the end RTLD_LOCAL make the linked > python.so not "reachable" from the python-efl modules. > > Are there some reasons to not use RTLD_GLOBAL in e_module_new? i spotted lots of modules with the same symbols. these get loaded into the global namespace of e and there is now only pure luck keeping things working as now we have multiple symbols with the same name and different memory locations. it was bad engineering to expose all those symbols. people will start to create symbols in one module then dlopen them or just call them in another without explicit dependency handling. there has to be namespacing... and one problem is all modules expose the SAME public symbols because it's their public api. we dlsym these from the module handle but they get loaded into a global table still. it's just luck that nothing broke. now one big reason is dependencies. for example lbpython2.7 vs libpython3.8 let's imagine we have 2 modules that use python as a scripting engine. 2 edgar like things. one links to python2.x and one to python3.x these libs break api ... BUT they share symbols they export. here's a list of some symbols that are the same: _Py_add_one_to_index_C _Py_add_one_to_index_F Py_AddPendingCall _Py_alias PyArena_AddPyObject PyArena_Free PyArena_Malloc PyArena_New _PyArg_NoKeywords PyArg_Parse _PyArg_Parse_SizeT PyArg_ParseTuple PyArg_ParseTupleAndKeywords _PyArg_ParseTupleAndKeywords_SizeT _PyArg_ParseTuple_SizeT _Py_arguments PyArg_UnpackTuple PyArg_VaParse ... there are over 780 of these symbols that conflict. so by e dlopening a module with GLOBAL instead of LOCAL... the libs the module links to automatically fill global symbol namespace with all of these symbols... if you have 2 modules that get these syms from different .so's - especially abi incompatible so's like libpython... then it's a matter of luck which symbol gets found first when a binding is loaded. that's why i moved to LOCAL... to stop these accidents. this can happen with 2 libjpegs that may be abi incompatible or 2 libgifs or libpngs or any library at all whre 2 non-identical versions may exist and different modules may implicitly bring them in as a linked dep... so this is why i might say it might be best to not link libpython and explicitly dlopen it and in this case be explicit about exposing symbols, or find some way of getting the python bindings and the python interpreter lib to bind together within a local namespace. e.g. dlopen or link to the right libpython interpreter and have the python bindings link also to the right python interpreter lib. then the bindings will bring in their own deps and find the right symbols in the right lib... and it'll be in local namespace for that module. > Il giorno ven 17 apr 2020 alle ore 23:56 Carsten Haitzler < > [email protected]> ha scritto: > > > On Fri, 17 Apr 2020 23:20:25 +0200 Davide Andreoli <[email protected] > > > > > said: > > > > > I found a way to get a bit more info, the python error while importing > > and > > > failing is: > > > ImportError: /usr/lib/python3.8/site-packages/efl/ > > > eo.cpython-38-x86_64-linux-gnu.so: undefined symbol: > > PyUnicode_FromFormat > > > > oooooh. ... so it is a symbol problem. :) That i believe is provided by > > libpython3.8.so ... edgar is linked to this... my guess is then python is > > dlopening the pytno-efl binding .so files and expecting the symbols to link > > up... but they don't. > > > > an idea. dlopen libpython3.8.so and use RTDL_GLOBAL to put python symbols > > in > > the global namespace instead of linking to libpython at compile time - do > > it > > runtime? then edgar can also choose at runtime either python 2 or python3 > > libs > > - dlsym the symbols you need from these... ? just trying the dlopen and > > stuffing it into global namespace may help... > > > > > eo.cpython-38-x86_64-linux-gnu.so is the python-efl module that bind > > the eo > > > library, > > > PyUnicode_FromFormat is a normal CPython function that should work... > > > the edgar module does not get correctly linked to python? > > > > > > Il giorno ven 17 apr 2020 alle ore 21:54 Carsten Haitzler < > > > [email protected]> ha scritto: > > > > > > > On Fri, 17 Apr 2020 19:33:30 +0200 Davide Andreoli < > > [email protected] > > > > > > > > > said: > > > > > > > > hmm no. removing RTLD_LOCAL makes no difference. so there goes that > > idea. i > > > > have python-efl bindings installed - i can run ecoonman-bin as a test. > > > > there is > > > > no output i see from python complaining... knowing what exactly fails > > will > > > > be > > > > key to finding out why. like cant find a symbol? missing > > link/dependency on > > > > load? something else? as there is nothing telling me what... it needs > > much > > > > deeper digging > > > > > > > > > Hi guys, > > > > > I'm lost in trying to fix the Edgar module to run again in E. Since > > some > > > > > months the python import machinery does not work anymore and the > > edgar > > > > > module is not able anymore to import the python efl bindings. > > > > > > > > > > Nothing has changed in Edgar nor in python-efl and I'm able to > > correctly > > > > > use the machinery in a simple test app (attached), it just does not > > work > > > > > when used inside an E module. I suspect something related to the new > > > > meson > > > > > build, but not sure at all. > > > > > > > > > > The simple test attached can be built as: > > > > > gcc -o test_py test_py.c `python-config --cflags --libs --embed` > > > > > `pkg-config --libs --cflags efl-ui` > > > > > when run it print OK, that means the python-efl module has been > > found and > > > > > linked. While the same code in edgar fails on the import_efl__eo() > > call. > > > > > Note the test also need the attached .h file to build > > > > > > > > > > To build/run this simple code of course python-efl must be installed > > > > > https://phab.enlightenment.org/w/projects/python_bindings_for_efl/ > > > > > > > > > > The edgar module instead is documented here: > > > > > https://phab.enlightenment.org/w/emodules/edgar/ > > > > > > > > > > Any help/ideas would be appreciated as I really don't have any idea > > atm > > > > > > > > > > The same issue has been recently raised also on the e-user mailing > > list > > > > > > > > > > Thanks > > > > > DaveMDS > > > > > > > > > > > > -- > > > > ------------- Codito, ergo sum - "I code, therefore I am" > > -------------- > > > > Carsten Haitzler - [email protected] > > > > > > > > > > > > > > -- > > ------------- Codito, ergo sum - "I code, therefore I am" -------------- > > Carsten Haitzler - [email protected] > > > > -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- Carsten Haitzler - [email protected] _______________________________________________ enlightenment-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
