eli-schwartz commented on PR #45854:
URL: https://github.com/apache/arrow/pull/45854#issuecomment-2751966330
> I think what ultimately fixed the segfault was building the arrow_python
library as a `py.extension_module` within the configuration and linking against
that. This seems to work on Linux, but on OSX Meson throws the following error:
Shared modules (dlopen plugins) don't have a SONAME field and in general the
linker isn't really meant to link directly to them, they are meant to be
dlopened. It's a deprecation warning on Linux.
I don't really understand why linking against an extension module would be
desirable or wanted. The way you're supposed to have an extension module
provide a C API for other modules to use is via capsules, as you can't
otherwise guarantee it gets loaded correctly.
It looks like arrow_python isn't even an extension module anyway. It's
common C++ code used by multiple extensions, which need to be loaded into the
same process at the same time. I'm not necessarily surprised that statically
linking a separate copy of the code into each extension could cause issues (the
segfault sounds plausibly like one extension is stomping on the memory of
another extension). Shared linking sounds like a reasonable solution, if that's
the case.
You can install the shared library to the default libdir -- /usr/lib64 when
simply using `meson install`. The meson-python backend should then relocate the
library into the wheel as `{site-packages}/.pyarrow.mesonpy.libs/*.so` and
update the extension modules with a suitable rpath to
`${ORIGIN}/../.pyarrow.mesonpy.libs`.
And because it's a standard shared library, there's only one copy of it in
the process space.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
