[
https://issues.apache.org/jira/browse/ARROW-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388345#comment-16388345
]
ASF GitHub Bot commented on ARROW-2269:
---------------------------------------
xhochy commented on issue #1713: ARROW-2269: [Python] Fixing paths for libs
when building bdist
URL: https://github.com/apache/arrow/pull/1713#issuecomment-370894072
@mitar When you build these wheels in your setup, do you actually have the
need to bundle Boost or would it be correctly installed on all target systems?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Cannot build bdist_wheel for Python
> -----------------------------------
>
> Key: ARROW-2269
> URL: https://issues.apache.org/jira/browse/ARROW-2269
> Project: Apache Arrow
> Issue Type: Bug
> Components: Packaging
> Affects Versions: 0.9.0
> Reporter: Mitar
> Priority: Major
> Labels: pull-request-available
>
> I am trying current master.
> I ran:
> {{{{
> python setup.py build_ext --build-type=$ARROW_BUILD_TYPE --with-parquet
> --with-plasma --bundle-arrow-cpp bdist_wheel
> }}}}
> Output:
> {{{{
> running build_ext
> creating build
> creating build/temp.linux-x86_64-3.6
> -- Runnning cmake for pyarrow
> cmake -DPYTHON_EXECUTABLE=.../Temp/arrow/pyarrow/bin/python
> -DPYARROW_BUILD_PARQUET=on -DPYARROW_BOOST_USE_SHARED=on
> -DPYARROW_BUILD_PLASMA=on -DPYARROW_BUNDLE_ARROW_CPP=ON
> -DCMAKE_BUILD_TYPE=release .../Temp/arrow/arrow/python
> -- The C compiler identification is GNU 7.2.0
> -- The CXX compiler identification is GNU 7.2.0
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> INFOCompiler command: /usr/bin/c++
> INFOCompiler version: Using built-in specs.
> COLLECT_GCC=/usr/bin/c++
> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
> OFFLOAD_TARGET_NAMES=nvptx-none
> OFFLOAD_TARGET_DEFAULT=1
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Ubuntu
> 7.2.0-8ubuntu3.2' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs
> --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr
> --with-gcc-major-version-only --program-suffix=-7
> --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id
> --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix
> --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes
> --with-default-libstdcxx-abi=new --enable-gnu-unique-object
> --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie
> --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto
> --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64
> --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic
> --enable-offload-targets=nvptx-none --without-cuda-driver
> --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu
> --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 7.2.0 (Ubuntu 7.2.0-8ubuntu3.2)
> INFOCompiler id: GNU
> Selected compiler gcc 7.2.0
> -- Performing Test CXX_SUPPORTS_SSE3
> -- Performing Test CXX_SUPPORTS_SSE3 - Success
> -- Performing Test CXX_SUPPORTS_ALTIVEC
> -- Performing Test CXX_SUPPORTS_ALTIVEC - Failed
> Configured for RELEASE build (set with cmake
> -DCMAKE_BUILD_TYPE={release,debug,...})
> -- Build Type: RELEASE
> -- Build output directory:
> .../Temp/arrow/arrow/python/build/temp.linux-x86_64-3.6/release/
> -- Found PythonInterp: .../Temp/arrow/pyarrow/bin/python (found version
> "3.6.3")
> -- Searching for Python libs in
> .../Temp/arrow/pyarrow/lib64;.../Temp/arrow/pyarrow/lib;/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu
> -- Looking for python3.6m
> -- Found Python lib
> /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.so
> -- Found PythonLibs:
> /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.so
> -- Found NumPy: version "1.14.1"
> .../Temp/arrow/pyarrow/lib/python3.6/site-packages/numpy/core/include
> -- Searching for Python libs in
> .../Temp/arrow/pyarrow/lib64;.../Temp/arrow/pyarrow/lib;/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu
> -- Looking for python3.6m
> -- Found Python lib
> /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.so
> -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
> -- Checking for module 'arrow'
> -- Found arrow, version 0.9.0-SNAPSHOT
> -- Arrow ABI version: 0.0.0
> -- Arrow SO version: 0
> -- Found the Arrow core library: .../Temp/arrow/dist/lib/libarrow.so
> -- Found the Arrow Python library: .../Temp/arrow/dist/lib/libarrow_python.so
> -- Boost version: 1.63.0
> -- Found the following Boost libraries:
> -- system
> -- filesystem
> -- regex
> Added shared library dependency arrow: .../Temp/arrow/dist/lib/libarrow.so
> Added shared library dependency arrow_python:
> .../Temp/arrow/dist/lib/libarrow_python.so
> -- Found the Parquet library: .../Temp/arrow/dist/lib/libparquet.so
> Added shared library dependency parquet: .../Temp/arrow/dist/lib/libparquet.so
> -- Checking for module 'plasma'
> -- Found plasma, version
> -- Plasma ABI version: 0.0.0
> -- Plasma SO version: 0
> -- Found the Plasma core library: .../Temp/arrow/dist/lib/libplasma.so
> -- Found Plasma executable: .../Temp/arrow/dist/bin/plasma_store
> Added shared library dependency libplasma:
> .../Temp/arrow/dist/lib/libplasma.so
> -- Configuring done
> -- Generating done
> -- Build files have been written to:
> .../Temp/arrow/arrow/python/build/temp.linux-x86_64-3.6
> -- Finished cmake for pyarrow
> -- Running cmake --build for pyarrow
> make
> Scanning dependencies of target lib_pyx
> [ 11%] Compiling Cython CXX source for lib...
> [ 11%] Built target lib_pyx
> Scanning dependencies of target lib
> [ 22%] Building CXX object CMakeFiles/lib.dir/lib.cxx.o
> [ 33%] Linking CXX shared module release/lib.cpython-36m-x86_64-linux-gnu.so
> [ 33%] Built target lib
> Scanning dependencies of target _parquet_pyx
> [ 44%] Compiling Cython CXX source for _parquet...
> [ 44%] Built target _parquet_pyx
> Scanning dependencies of target _parquet
> [ 55%] Building CXX object CMakeFiles/_parquet.dir/_parquet.cxx.o
> [ 66%] Linking CXX shared module
> release/_parquet.cpython-36m-x86_64-linux-gnu.so
> [ 66%] Built target _parquet
> Scanning dependencies of target plasma_pyx
> [ 77%] Compiling Cython CXX source for plasma...
> [ 77%] Built target plasma_pyx
> Scanning dependencies of target plasma
> [ 88%] Building CXX object CMakeFiles/plasma.dir/plasma.cxx.o
> [100%] Linking CXX shared module
> release/plasma.cpython-36m-x86_64-linux-gnu.so
> [100%] Built target plasma
> -- Finished cmake --build for pyarrow
> .../Temp/arrow/arrow/python/build/lib.linux-x86_64-3.6/pyarrow
> ['release/libarrow.so.0.0.0', 'release/libarrow.so.0', 'release/libarrow.so']
> release/libarrow.so.0.0.0
> ['release/libarrow_python.so.0.0.0', 'release/libarrow_python.so.0',
> 'release/libarrow_python.so']
> release/libarrow_python.so.0.0.0
> ['release/libplasma.so.0.0.0', 'release/libplasma.so.0',
> 'release/libplasma.so']
> release/libplasma.so.0.0.0
> ['release/libparquet.so.1.3.2', 'release/libparquet.so.1',
> 'release/libparquet.so']
> release/libparquet.so.1.3.2
> }}}}
> And after that it crashes because it cannot find {{arrow_boost_filesystem}}
> in {{build_prefix}} in {{_move_shared_libs_unix}} call which makes {{libs}}
> variable there empty.
> If I do:
> {{{{
> $ find . -name *.so
> ./build/lib.linux-x86_64-3.6/pyarrow/libparquet.so
> ./build/lib.linux-x86_64-3.6/pyarrow/libplasma.so
> ./build/lib.linux-x86_64-3.6/pyarrow/libarrow.so
> ./build/lib.linux-x86_64-3.6/pyarrow/libarrow_python.so
> ./build/temp.linux-x86_64-3.6/release/_parquet.cpython-36m-x86_64-linux-gnu.so
> ./build/temp.linux-x86_64-3.6/release/libparquet.so
> ./build/temp.linux-x86_64-3.6/release/libboost_filesystem.so
> ./build/temp.linux-x86_64-3.6/release/libplasma.so
> ./build/temp.linux-x86_64-3.6/release/libarrow.so
> ./build/temp.linux-x86_64-3.6/release/libboost_system.so
> ./build/temp.linux-x86_64-3.6/release/lib.cpython-36m-x86_64-linux-gnu.so
> ./build/temp.linux-x86_64-3.6/release/libboost_regex.so
> ./build/temp.linux-x86_64-3.6/release/libarrow_python.so
> ./build/temp.linux-x86_64-3.6/release/plasma.cpython-36m-x86_64-linux-gnu.so
> ./pyarrow/_parquet.cpython-36m-x86_64-linux-gnu.so
> ./pyarrow/lib.cpython-36m-x86_64-linux-gnu.so
> ./pyarrow/plasma.cpython-36m-x86_64-linux-gnu.so
> }}}}
> So it seems the issue is that the filename is {{libboost_filesystem.so}}, but
> it is trying to move {{libarrow_boost_filesystem.so}}?
> Changing lines in {{setup.py}} to:
> {{{{
> move_shared_libs(build_prefix, build_lib,
> "boost_filesystem")
> move_shared_libs(build_prefix, build_lib, "boost_system")
> move_shared_libs(build_prefix, build_lib, "boost_regex")
> }}}}
> seems to make it build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
