I don't know if it's helpful, but [1] is how I address the error you get regarding the compile symbol "_ZNK5arrow6Schema8ToStringB5cxx11Eb". For context, I build arrow from source, use arrow from C++, and I use arrow from python (pyarrow and my own cython that calls my C++ code). I also use poetry to manage my python environments, so when I use pip to install pyarrow (but recompiling and pointing to the arrow_python library), I also make sure to do it from my poetry environment.
I realized at some point that my pyarrow cython lib wasn't pointing to the right dependency so it took awhile to figure out, but in [1] I have included a file, "check-pyarrow-deps.bash", that should show how I figured out this problem via the "ldd" command (I'm pretty sure that's what I used). Not sure how to fix your other problem. [1] https://gist.github.com/drin/5dbda4aa546c3bf4a0058cd1402d5b4d Aldrin Montana Computer Science PhD Student UC Santa Cruz On Mon, Sep 20, 2021 at 1:02 PM Weston Pace <[email protected]> wrote: > You might want to take a look at [1] which has some discussion from a > previous (unfortunately, ultimately unsuccessful) attempt at getting > pybind11 to work. If you give us a high level overview of what you > are trying to achieve we may also be able to suggest a better > solution. For example, you may be able to achieve what you want using > the C data interface [2] which is going to be more stable. > > If you want to use pybind11 however you will need to ensure that the C > & C++ libraries (e.g. glibc / libstdc++) used to build the binary > shared objects shipped with pyarrow are the same as the libraries used > to build your own projects. My guess is that the error you are seeing > in (2) is a binary incompatibility between libstdc++ versions. > > Since you are installing pyarrow packages from PIP you may want to > read up on manylinux, which is PIP's attempt to handle this challenge. > Pyarrow's PIP wheels are manylinux wheels. Pyarrow distributes both a > manylinux2010 [3] wheel which supports: > > GLIBC_2.12 > CXXABI_1.3.3 > GLIBCXX_3.4.13 > GCC_4.5.0 > > ...and a manylinux2014 [4] wheel which supports: > > GLIBC_2.17 > CXXABI_1.3.7, CXXABI_TM_1 is also allowed > GLIBCXX_3.4.19 > GCC_4.8.0 > > I hope this helps get you started. > > [1] https://github.com/apache/arrow/issues/10488 > [2] https://arrow.apache.org/docs/format/CDataInterface.html > [3] https://www.python.org/dev/peps/pep-0571/ > [4] https://www.python.org/dev/peps/pep-0599/ > > On Mon, Sep 20, 2021 at 9:04 AM Arthur Andres <[email protected]> > wrote: > > > > Hello all, > > > > I want to use pybind11 with arrow, but I can't get it to work. I get an > "undefined symbol" error at run time. > > > > I've put a simple version of what I'm trying to do on github: > https://github.com/0x26res/pyarrowbind > > > > In a nutshell, I'm trying to follow the instructions here: > https://arrow.apache.org/docs/python/extending.html#c-api > > * I have a virtual environment with pybind, pyarrow (and numpy) installed > > * I've made sure I've called `pa.create_library_symlinks()` > > * I'm linking against libarrow_python.so and libarrow.so > > > > Here's how far I got to far: > > 1. If my C++ code only tries to call `arrow::py::import_pyarrow()`, it > works (and returns 0) > > 2. If my C++ code tries to do anything else with arrow (like creating a > schema), I get this error: > > ``` > > Traceback (most recent call last): > > File "<string>", line 1, in <module> > > ImportError: /home/arthur/source/pyarrowbind/build/ > bindings.cpython-38-x86_64-linux-gnu.so: undefined symbol: > _ZNK5arrow6Schema8ToStringB5cxx11Eb > > ``` > > > > 3. I tried to link against the "arrow_static" library, provided by > `FIND_PACKAGE(Arrow REQUIRED)` in cmake. When I do that I get a different > error: > > ``` > > Traceback (most recent call last): > > File "<string>", line 1, in <module> > > ImportError: /home/arthur/source/pyarrowbind/build/ > bindings.cpython-38-x86_64-linux-gnu.so: undefined symbol: > ERR_error_string_n > > ``` > > This is pointing to a symbol defined in openssl. I tried to fix this, > then got a similar error with curl symbols, which I fixed and eventually > got a segfault. Which makes me think I must be doing something wrong. > > > > > > Do you have any idea about what I'm doing wrong? > > > > I'm using: > > Ubuntu 20.04.3 LTS > > Python 3.8.10 > > pyarrow==5.0.0 > > pybind11==2.7.1 > > numpy==1.21.2 > > > > Thanks. >
