Hi Sutou, Thanks for your help. I tried that option but unfortunately arrow build scripts can't find the libarrow installation from vcpkg it seems to be using a custom findArrow.cmake script. Anyway I hacked the CMakeLists from arrow in order to find vcpkg's arrow libraries and it worked, I run ldd and the libraries are the ones from vcpkg. So now I have my Python extension and my application linked against vcpkg's arrow and arrow_python. But now it still doesn't work I am observing a couple of weird issues. 1.- In the python script that my embedded python interpreter runs I do: "from statsmodels.tsa.stattools import adfuller as adf" but it gets stuck there. So I removed that import. Then: 2.- After removing the import now I have a SIGABRT and the following message when my extensions calls "arrow::py::wrap_array": "Fatal Python error: PyThreadState_Get: no current thread". I did some Google and I only found a similar issue on Macs with different python interpreters installed, but I am on an Ubuntu 18.4 with only python 3.6.9 installed and I am using a venv for pyarrow. I don't know what to do next. Did any of you experienced something similar?
Thanks! Raúl Bocanegra Algarra Software Engineer M: + 34 617 83 64 45 - E: [email protected]<mailto:[email protected]> ________________________________ From: Sutou Kouhei <[email protected]> Sent: Sunday, January 5, 2020 10:29 PM To: [email protected] <[email protected]> Subject: Re: Using Pyarrow and C++ API Hi, How about install pyarrow with "pip install --no-binary :all: pyarrow"? Then you will be able to build your pyarrow with your libarrow.so and libarrow_python.so. Thanks, -- kou In <am0pr01mb641703d6ba43cf50596dfd34e7...@am0pr01mb6417.eurprd01.prod.exchangelabs.com> "Using Pyarrow and C++ API " on Sun, 5 Jan 2020 03:45:21 +0000, Raúl Bocanegra Algarra <[email protected]> wrote: > Hi! > > I am trying to use pyarrow with arrow C++ API in an application that embeds a > python3 interpreter and loads an extension module using pybind11. > Documentation says C++ headers and libraries are bundled with pyarrow but I > am having some segfaults when calling some API functions like the wrap/unwrap > ones. I am calling import_pyarrow and also import_numpy but segfaults still > happening. I feel the reason is that I compile and link with my own arrow and > arrow_python libs built with vcpkg so my app links with those, but the > extension module imported by the embedded python interpreter is loading the > arrow_python from the site-packages folder where pip installed pyarrow, and > that mismatch makes the segfault happen. So I was wondering if the correct > approach for a situation like this with an embedded interpreter and an > extension module that imports pyarrow is to use the headers and libs from the > pyarrow installation removing the ones from vcpkg or if you know another > option I haven't contemplated yet. > > Thanks for your work. > > Best regards, > > Raúl Bocanegra Algarra. C++ Software Engineer. >
