Thanks for the response! I'll try that out. It didn't occur to me that archlinux might be building the static libraries yet not installing them (and/or removing them).
I'll check a few things and report back here what works. Aldrin Montana Computer Science PhD Student UC Santa Cruz On Fri, Feb 11, 2022 at 6:43 PM David Li <lidav...@apache.org> wrote: > Hmm, interesting. So basically you want a PyArrow built against the system > libarrow? > > That error is CMake complaining that the CMake files for libarrow were > generated referencing static libraries, but that the actual files aren't > there. Indeed, looking at the Arch Linux package, the static libraries are > missing, except for libarrow_bundled_dependencies. > > But Arrow does build the static libraries by default. Is there something > that removes them after build, as part of packaging? (Indeed, it seems > makepkg removes them by default if there's an equivalent shared library > [1], that would explain why libarrow_bundled_dependencies is there but > nothing else is.) Perhaps try updating that PKGBUILD to also specify > -DARROW_BUILD_STATIC=OFF; then CMake shouldn't generate arrow_static > targets, and then CMake won't complain later. (I haven't yet tried this, > though.) And as a bonus, you should save a bit of build time. > > [1]: https://archlinux.org/pacman/PKGBUILD.5.html > > -David > > On Fri, Feb 11, 2022, at 19:45, Aldrin wrote: > > Hello! > > > > > > *tl;dr. *I want to figure out how to build pyarrow from source (using > pip) > > and tell it *not* to use static libraries (libarrow.a). Can anyone tell > me > > if there's a CMAKE flag that the setup.py can propagate, or an option > that > > tells setup.py not to look for static libraries? I tried searching the > > current version and nothing I've tried seems to have worked. > > > > > > > > *----* > > *Long Version.* > > > > For context, I recompile the pyarrow package from source because > installing > > it from binaries causes it to have different symbols from the arrow > > libraries I have installed. I'm not 100% positive on the mechanism here, > > but I suspect that the following command basically just uses my system's > > various toolchains to recompile pyarrow so that library symbols match: > > > > # where ArchLinux installs it > > arrow_pythonopt="-DArrowPython_DIR=/usr/lib/cmake/arrow" > > arrow_staticopt="-DARROW_BUILD_STATIC=OFF" > > > > # so that we can install a re-compiled binary > > export PYARROW_CMAKE_OPTIONS="${arrow_pythonopt} ${arrow_staticopt}" > > python3 -m pip install --no-binary :all: pyarrow==6.0.1 > > > > I had been doing this since Arrow 5.0.0 and it seems to have stopped with > > some update from the 7.0.0 release. Now, the problem is that when I > > recompile pyarrow using the above command, I get the following error: > > > > -- Found PkgConfig: /usr/bin/pkgconf (found version "1.8.0") > > CMake Error at /usr/lib64/cmake/arrow/ArrowTargets.cmake:96 (message): > > The imported target "arrow_static" references the file > > > > "/usr/lib/libarrow.a" > > > > I am on Archlinux: > > > > 5.15.4-arch1-1 > > > > and the system package builds arrow like so (defined here > > < > https://github.com/archlinux/svntogit-community/blob/packages/arrow/trunk/PKGBUILD > > > > ): > > > > (Some options omitted inline because I think they're irrelevant) > > > > ARROW_BUILD_TOOLCHAIN=/usr \ > > ORC_HOME=/usr \ > > DOUBLE_CONVERSION_HOME=/usr \ > > cmake \ > > -B build \ > > -S apache-${pkgname}-${pkgver}/cpp \ > > -DCMAKE_INSTALL_PREFIX="/usr" \ > > -DCMAKE_INSTALL_LIBDIR="lib" \ > > -DCMAKE_BUILD_TYPE=Release \ > > -DARROW_DEPENDENCY_SOURCE=SYSTEM \ > > -DARROW_BUILD_TESTS=ON \ > > -DARROW_COMPUTE=ON \ > > -DARROW_GANDIVA=OFF \ > > -DARROW_PYTHON=ON \ > > -DARROW_SIMD_LEVEL=AVX2 \ > > -DARROW_USE_GLOG=ON \ > > -DARROW_WITH_BROTLI=ON \ > > -DPARQUET_REQUIRE_ENCRYPTION=ON > > make -C build > > > > > > > > Thank you for any help you can offer! > > > > Aldrin Montana > > Computer Science PhD Student > > UC Santa Cruz >
