Hi, Changing
> RUN python3 -c 'import pyarrow' to RUN LD_LIBRARY_PATH=/usr/local/lib python3 -c 'import pyarrow' works on my environment. Other solution: Adding ENV LD_LIBRARY_PATH=/usr/local/lib before RUN python3 -c 'import pyarrow' Dockerfile: ... RUN bash install_arrow.sh ENV LD_LIBRARY_PATH=/usr/local/lib RUN python3 -c 'import pyarrow' Thanks, -- kou In <CALWsJGTCEvKH=nj_bn9hescrnunqgphgedwsvhaa1xptsmv...@mail.gmail.com> "Pyarrow build/install from source in ubuntu not working" on Thu, 23 Jan 2020 17:38:33 -0800, Anna Waldron <[email protected]> wrote: > Hi, > > I am trying to build and install pyarrow from source in an ubuntu 18.04 > docker image and getting the following error when attempting to import the > module: > > Traceback (most recent call last): >> File "<string>", line 1, in <module> >> File >> "/usr/local/lib/python3.6/dist-packages/pyarrow-0.14.0-py3.6-linux-x86_64.egg/pyarrow/__init__.py", >> line 49, in <module> >> from pyarrow.lib import cpu_count, set_cpu_count >> ImportError: libarrow.so.14: cannot open shared object file: No such file >> or directory >> > > Here is the Dockerfile I am using: > > FROM ubuntu:18.04 >> RUN apt-get update >> RUN apt-get install -y git >> RUN mkdir /arrow >> RUN git clone https://github.com/apache/arrow.git /arrow >> WORKDIR /arrow/arrow >> RUN git checkout apache-arrow-0.14.0 >> WORKDIR / > > COPY install_arrow.sh /install_arrow.sh > > RUN bash install_arrow.sh > > >> RUN python3 -c 'import pyarrow' > > > and the install_arrow.sh script copied into the image: > > export ARROW_BUILD_TYPE=release >> export ARROW_HOME=/usr/local \ >> PARQUET_HOME=/usr/local >> export PYTHON_EXECUTABLE=/usr/bin/python3 >> > > > # install requirements >> export DEBIAN_FRONTEND="noninteractive" >> apt-get update >> apt-get install -y --no-install-recommends apt-utils >> apt-get install -y git python3-minimal python3-pip autoconf libtool >> apt-get install -y cmake \ >> python3-dev \ >> libjemalloc-dev libboost-dev \ >> build-essential \ >> libboost-filesystem-dev \ >> libboost-regex-dev \ >> libboost-system-dev \ >> flex \ >> bison >> pip3 install --no-cache-dir six pytest numpy cython > > >> mkdir -p /arrow/cpp/build \ >> && cd /arrow/cpp/build \ >> && cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE \ >> -DOPENSSL_ROOT_DIR=/usr/local/ssl \ >> -DCMAKE_INSTALL_LIBDIR=lib \ >> -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \ >> -DARROW_PARQUET=ON \ >> -DARROW_PYTHON=ON \ >> -DARROW_PLASMA=ON \ >> -DARROW_BUILD_TESTS=OFF \ >> -DPYTHON_EXECUTABLE=$PYTHON_EXECUTABLE \ >> .. \ >> && make -j$(nproc) \ >> && make install \ >> && cd /arrow/python \ >> && python3 setup.py build_ext --build-type=$ARROW_BUILD_TYPE >> --with-parquet \ >> && python3 setup.py install >> > > > LD_LIBRARY_PATH=/usr/local/lib >> > > I'm using Docker 19.03.5 on Ubuntu 18.04.3 LTS to build the image. > > Thanks in advance for any help. > > Anna
