How did you install pyarrow, with pip? It has its own C++ libraries bundled inside and so will surely conflict with the vcpkg libraries
On Tue, Jan 7, 2020 at 11:18 AM Raúl Bocanegra Algarra <[email protected]> wrote: > > Hi Wes, > > I just added "-DARROW_PYTHON=ON" on vcpkg/ports/arrow/portfile.cmake on the > "vcpkg_configure_cmake" function call. > I also created an improvement for this in its repo: > https://github.com/microsoft/vcpkg/issues/9350 > Also edited its vcpkg/triplets/x64-linux.cmake as they recommend on the docs: > https://vcpkg.readthedocs.io/en/latest/users/triplets/#per-port-customization > Just edited: > set(VCPKG_LIBRARY_LINKAGE static) > if(PORT MATCHES "arrow") > set(VCPKG_LIBRARY_LINKAGE dynamic) > endif() > > Thanks for your work. > > Raúl. > ________________________________ > From: Wes McKinney <[email protected]> > Sent: Tuesday, January 7, 2020 6:06:21 PM > To: [email protected] <[email protected]> > Cc: Sutou Kouhei <[email protected]> > Subject: Re: Using Pyarrow and C++ API > > It doesn't seem like vcpkg should have libarrow_python available based on > > https://github.com/microsoft/vcpkg/tree/master/ports/arrow > > How are you installing pyarrow? > > On Tue, Jan 7, 2020 at 10:14 AM Raúl Bocanegra Algarra > <[email protected]> wrote: > > > > Hi Sutou, > > > > Thanks for your help. I tried that option but unfortunately arrow build > > scripts can't find the libarrow installation from vcpkg it seems to be > > using a custom findArrow.cmake script. Anyway I hacked the CMakeLists from > > arrow in order to find vcpkg's arrow libraries and it worked, I run ldd and > > the libraries are the ones from vcpkg. So now I have my Python extension > > and my application linked against vcpkg's arrow and arrow_python. But now > > it still doesn't work I am observing a couple of weird issues. > > 1.- In the python script that my embedded python interpreter runs I do: > > "from statsmodels.tsa.stattools import adfuller as adf" but it gets stuck > > there. So I removed that import. Then: > > 2.- After removing the import now I have a SIGABRT and the following > > message when my extensions calls "arrow::py::wrap_array": > > "Fatal Python error: PyThreadState_Get: no current thread". > > I did some Google and I only found a similar issue on Macs with different > > python interpreters installed, but I am on an Ubuntu 18.4 with only python > > 3.6.9 installed and I am using a venv for pyarrow. > > I don't know what to do next. Did any of you experienced something similar? > > > > Thanks! > > > > > > Raúl Bocanegra Algarra > > > > Software Engineer > > > > M: + 34 617 83 64 45 - E: [email protected] > > > > > > > > ________________________________ > > From: Sutou Kouhei <[email protected]> > > Sent: Sunday, January 5, 2020 10:29 PM > > To: [email protected] <[email protected]> > > Subject: Re: Using Pyarrow and C++ API > > > > Hi, > > > > How about install pyarrow with "pip install --no-binary :all: pyarrow"? > > Then you will be able to build your pyarrow with your > > libarrow.so and libarrow_python.so. > > > > Thanks, > > -- > > kou > > > > In > > > > <am0pr01mb641703d6ba43cf50596dfd34e7...@am0pr01mb6417.eurprd01.prod.exchangelabs.com> > > "Using Pyarrow and C++ API " on Sun, 5 Jan 2020 03:45:21 +0000, > > Raúl Bocanegra Algarra <[email protected]> wrote: > > > > > Hi! > > > > > > I am trying to use pyarrow with arrow C++ API in an application that > > > embeds a python3 interpreter and loads an extension module using > > > pybind11. Documentation says C++ headers and libraries are bundled with > > > pyarrow but I am having some segfaults when calling some API functions > > > like the wrap/unwrap ones. I am calling import_pyarrow and also > > > import_numpy but segfaults still happening. I feel the reason is that I > > > compile and link with my own arrow and arrow_python libs built with vcpkg > > > so my app links with those, but the extension module imported by the > > > embedded python interpreter is loading the arrow_python from the > > > site-packages folder where pip installed pyarrow, and that mismatch makes > > > the segfault happen. So I was wondering if the correct approach for a > > > situation like this with an embedded interpreter and an extension module > > > that imports pyarrow is to use the headers and libs from the pyarrow > > > installation removing the ones from vcpkg or if you know another option I > > > haven't contemplated yet. > > > > > > Thanks for your work. > > > > > > Best regards, > > > > > > Raúl Bocanegra Algarra. C++ Software Engineer. > > >
