Hi Wes, I just added "-DARROW_PYTHON=ON" on vcpkg/ports/arrow/portfile.cmake on the "vcpkg_configure_cmake" function call. I also created an improvement for this in its repo: https://github.com/microsoft/vcpkg/issues/9350 Also edited its vcpkg/triplets/x64-linux.cmake as they recommend on the docs: https://vcpkg.readthedocs.io/en/latest/users/triplets/#per-port-customization Just edited: set(VCPKG_LIBRARY_LINKAGE static) if(PORT MATCHES "arrow") set(VCPKG_LIBRARY_LINKAGE dynamic) endif()
Thanks for your work. Raúl. ________________________________ From: Wes McKinney <[email protected]> Sent: Tuesday, January 7, 2020 6:06:21 PM To: [email protected] <[email protected]> Cc: Sutou Kouhei <[email protected]> Subject: Re: Using Pyarrow and C++ API It doesn't seem like vcpkg should have libarrow_python available based on https://github.com/microsoft/vcpkg/tree/master/ports/arrow How are you installing pyarrow? On Tue, Jan 7, 2020 at 10:14 AM Raúl Bocanegra Algarra <[email protected]> wrote: > > Hi Sutou, > > Thanks for your help. I tried that option but unfortunately arrow build > scripts can't find the libarrow installation from vcpkg it seems to be using > a custom findArrow.cmake script. Anyway I hacked the CMakeLists from arrow in > order to find vcpkg's arrow libraries and it worked, I run ldd and the > libraries are the ones from vcpkg. So now I have my Python extension and my > application linked against vcpkg's arrow and arrow_python. But now it still > doesn't work I am observing a couple of weird issues. > 1.- In the python script that my embedded python interpreter runs I do: > "from statsmodels.tsa.stattools import adfuller as adf" but it gets stuck > there. So I removed that import. Then: > 2.- After removing the import now I have a SIGABRT and the following message > when my extensions calls "arrow::py::wrap_array": > "Fatal Python error: PyThreadState_Get: no current thread". > I did some Google and I only found a similar issue on Macs with different > python interpreters installed, but I am on an Ubuntu 18.4 with only python > 3.6.9 installed and I am using a venv for pyarrow. > I don't know what to do next. Did any of you experienced something similar? > > Thanks! > > > Raúl Bocanegra Algarra > > Software Engineer > > M: + 34 617 83 64 45 - E: [email protected] > > > > ________________________________ > From: Sutou Kouhei <[email protected]> > Sent: Sunday, January 5, 2020 10:29 PM > To: [email protected] <[email protected]> > Subject: Re: Using Pyarrow and C++ API > > Hi, > > How about install pyarrow with "pip install --no-binary :all: pyarrow"? > Then you will be able to build your pyarrow with your > libarrow.so and libarrow_python.so. > > Thanks, > -- > kou > > In > > <am0pr01mb641703d6ba43cf50596dfd34e7...@am0pr01mb6417.eurprd01.prod.exchangelabs.com> > "Using Pyarrow and C++ API " on Sun, 5 Jan 2020 03:45:21 +0000, > Raúl Bocanegra Algarra <[email protected]> wrote: > > > Hi! > > > > I am trying to use pyarrow with arrow C++ API in an application that embeds > > a python3 interpreter and loads an extension module using pybind11. > > Documentation says C++ headers and libraries are bundled with pyarrow but I > > am having some segfaults when calling some API functions like the > > wrap/unwrap ones. I am calling import_pyarrow and also import_numpy but > > segfaults still happening. I feel the reason is that I compile and link > > with my own arrow and arrow_python libs built with vcpkg so my app links > > with those, but the extension module imported by the embedded python > > interpreter is loading the arrow_python from the site-packages folder where > > pip installed pyarrow, and that mismatch makes the segfault happen. So I > > was wondering if the correct approach for a situation like this with an > > embedded interpreter and an extension module that imports pyarrow is to use > > the headers and libs from the pyarrow installation removing the ones from > > vcpkg or if you know another option I haven't contemplated yet. > > > > Thanks for your work. > > > > Best regards, > > > > Raúl Bocanegra Algarra. C++ Software Engineer. > >
