I had so much fun with the wheels in the past, I'm now a happy member of conda-forge core instead :D
The good thing first: * The C++ ABI didn't change between the manylinux versions, it is the old one in all cases. So you mix & match manylinux versions. The sad things: * The manylinuxX standard are intented to provide a way to ship *self-contained* wheels that run on any recent Linux. The important part here is that they need to be self-contained. Having a binary dependency on another wheel is actually not allowed. * Thus the snowflake-python-connector ships the libarrow.so it was build with as part of its wheel. In this case auditwheel is happy with the wheel. * It is working with numpy as a dependency because NumPy linkage is similar to the import lib behaviour on Windows: You don't actually link against numpy but you statically link a set of functions that are resolved to NumPy's function when you import numpy. Quick googling leads to https://github.com/yugr/Implib.so which could provide something similar for Linux. * You could actually omit linking to libarrow and try to populate the symbols before you load the library. This is how the Python symbols are available to extensions without linking to libpython. On Thu, Jul 2, 2020, at 2:43 PM, Maarten Breddels wrote: > Ok, thanks! > > I'm setting up a repo with an example here, using pybind11: > https://github.com/vaexio/vaex-arrow-ext > > and I'll just try all possible combinations and report back. > > cheers, > > Maarten Breddels > Software engineer / consultant / data scientist > Python / C++ / Javascript / Jupyter > www.maartenbreddels.com / vaex.io > maartenbredd...@gmail.com +31 6 2464 0838 <+31+6+24640838> > [image: Twitter] <https://twitter.com/maartenbreddels>[image: Github] > <https://github.com/maartenbreddels>[image: LinkedIn] > <https://linkedin.com/in/maartenbreddels>[image: Skype] > > > > > Op do 2 jul. 2020 om 14:32 schreef Joris Van den Bossche < > jorisvandenboss...@gmail.com>: > > > Also no concrete answer, but one such example is turbodbc, I think. > > But it seems they only have conda binary packages, and don't > > distribute wheels .. > > (https://turbodbc.readthedocs.io/en/latest/pages/getting_started.html), > > so not that relevant as comparison (they also need to build against an > > odbc driver in addition to arrow). > > But maybe Uwe has some more experience in this regard (and with > > attempts building wheels for turbodbc, eg > > https://github.com/blue-yonder/turbodbc/pull/108). > > > > Joris > > > > On Thu, 2 Jul 2020 at 11:05, Antoine Pitrou <anto...@python.org> wrote: > > > > > > > > > Hi Maarten, > > > > > > Le 02/07/2020 à 10:53, Maarten Breddels a écrit : > > > > > > > > Also, I see pyarrow distributes manylinux1/2010/2014 wheels. Would a > > vaex > > > > extension distributed as a 2010 wheel, and build with the pyarrow 2010 > > > > wheel, work in an environment where someone installed a pyarrow 2014 > > > > wheel, or build from source, or installed from conda-forge? > > > > > > I have no idea about the concrete answer, but it probably depends > > > whether the libstdc++ ABI changed between those two versions. I'm > > > afraid you'll have to experiment yourself. > > > > > > (if you want to eschew C++ ABI issues, you may use the C Data Interface: > > > https://arrow.apache.org/docs/format/CDataInterface.html > > > though of course you won't have access to all the useful helpers in the > > > Arrow C++ library) > > > > > > Regards > > > > > > Antoine. > > > > > > > > >
