I can confirm what Uwe said, manylinux doesn't cause issues.
Here I've build inside a manylinux2010 docker a C++ Python extension (using
the C++ of Arrow):
https://github.com/vaexio/vaex-arrow-ext/runs/831763024?check_suite_focus=true
It's built with the manylinux1 and manylinux2010 pyarrow wheel
We build pyarrow in the docker image because auditwheel complains about pyarrow
otherwise which causes our wheels to fail auditwheel and not allow the
manylinux tag. But assuming we build pyarrow in the docker image, our manylinux
wheels that result are then compatible with the pyarrow manylinux
I did try the approach to not link against pyarrow but leave out the symbols,
just ensure pyarrow is imported before the vaex extension. This works
out-of-the-box on macOS but fails on Linux as symbols have a scope there.
Adding the following lines to load Arrow into the global scope made it wor
Hello Tim,
thanks for the hint. I see that you build arrow by yourselves in the
Dockerfile. Could it be that in the end you statically link the arrow libraries?
As there are no wheel on PyPI, I couldn't verify whether that assumption is
true.
Best
Uwe
On Thu, Jul 2, 2020, at 4:53 PM, Tim Pain
We spent a ton of time on this for perspective, the end result is a mostly
compatible set of wheels for most platforms, I believe we skipped py2 but
nobody cares about those anyway. We link against libarrow and libarrow_python
on Linux, on windows we vendor them all into our library. Feel free t
I had so much fun with the wheels in the past, I'm now a happy member of
conda-forge core instead :D
The good thing first:
* The C++ ABI didn't change between the manylinux versions, it is the old one
in all cases. So you mix & match manylinux versions.
The sad things:
* The manylinuxX standa
Ok, thanks!
I'm setting up a repo with an example here, using pybind11:
https://github.com/vaexio/vaex-arrow-ext
and I'll just try all possible combinations and report back.
cheers,
Maarten Breddels
Software engineer / consultant / data scientist
Python / C++ / Javascript / Jupyter
www.maartenb
Also no concrete answer, but one such example is turbodbc, I think.
But it seems they only have conda binary packages, and don't
distribute wheels ..
(https://turbodbc.readthedocs.io/en/latest/pages/getting_started.html),
so not that relevant as comparison (they also need to build against an
odbc d
Hi Maarten,
Le 02/07/2020 à 10:53, Maarten Breddels a écrit :
>
> Also, I see pyarrow distributes manylinux1/2010/2014 wheels. Would a vaex
> extension distributed as a 2010 wheel, and build with the pyarrow 2010
> wheel, work in an environment where someone installed a pyarrow 2014
> wheel, or
Hi,
again, in the process of adopting Arrow in Vaex, we need to have some
legacy c++ code in Vaex itself, and we might want to add some new functions
in c++ that might not be suitable for core Apache Arrow, or we need to ship
ourselves due to time constraints.
I am a bit worried about the C++ ABI
10 matches
Mail list logo
