[
https://issues.apache.org/jira/browse/ARROW-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999598#comment-15999598
]
Devang Shah commented on ARROW-955:
-----------------------------------
Yes! Thanks a million. However, a couple of tests fail:
(code)
-- Found the Parquet library: /usr/local/lib/libparquet.so
-- Found the Parquet Arrow library: /usr/local/lib
-- Found the Arrow core library:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow.so
-- Found the Arrow Python library:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_python.so
-- Found the Arrow jemalloc library:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_jemalloc.so
Added shared library dependency arrow:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow.so
Added shared library dependency arrow_python:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_python.so
Added shared library dependency parquet_arrow:
/usr/local/lib/libparquet_arrow.so
Added shared library dependency arrow_jemalloc:
/home/derdo/miniconda2/envs/pyarrow-dev/lib/libarrow_jemalloc.so
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/derdo/repos/arrow/python/build/temp.linux-x86_64-2.7
make
Scanning dependencies of target _parquet_pyx
[ 4%] Compiling Cython CXX source for _parquet...
[ 4%] Built target _parquet_pyx
Scanning dependencies of target _parquet
[ 8%] Building CXX object CMakeFiles/_parquet.dir/_parquet.cxx.o
[ 12%] Linking CXX shared module release/_parquet.so
[ 12%] Built target _parquet
Scanning dependencies of target _error_pyx
[ 16%] Compiling Cython CXX source for _error...
[ 16%] Built target _error_pyx
Scanning dependencies of target _error
[ 20%] Building CXX object CMakeFiles/_error.dir/_error.cxx.o
[ 25%] Linking CXX shared module release/_error.so
[ 25%] Built target _error
Scanning dependencies of target _jemalloc_pyx
[ 29%] Compiling Cython CXX source for _jemalloc...
[ 29%] Built target _jemalloc_pyx
Scanning dependencies of target _jemalloc
[ 33%] Building CXX object CMakeFiles/_jemalloc.dir/_jemalloc.cxx.o
[ 37%] Linking CXX shared module release/_jemalloc.so
[ 37%] Built target _jemalloc
Scanning dependencies of target _table_pyx
[ 41%] Compiling Cython CXX source for _table...
[ 41%] Built target _table_pyx
Scanning dependencies of target _table
[ 45%] Building CXX object CMakeFiles/_table.dir/_table.cxx.o
[ 50%] Linking CXX shared module release/_table.so
[ 50%] Built target _table
Scanning dependencies of target _config_pyx
[ 54%] Compiling Cython CXX source for _config...
[ 54%] Built target _config_pyx
Scanning dependencies of target _config
[ 58%] Building CXX object CMakeFiles/_config.dir/_config.cxx.o
[ 62%] Linking CXX shared module release/_config.so
[ 62%] Built target _config
Scanning dependencies of target _memory_pyx
[ 66%] Compiling Cython CXX source for _memory...
[ 66%] Built target _memory_pyx
Scanning dependencies of target _memory
[ 70%] Building CXX object CMakeFiles/_memory.dir/_memory.cxx.o
[ 75%] Linking CXX shared module release/_memory.so
[ 75%] Built target _memory
Scanning dependencies of target _array_pyx
[ 79%] Compiling Cython CXX source for _array...
[ 79%] Built target _array_pyx
Scanning dependencies of target _array
[ 83%] Building CXX object CMakeFiles/_array.dir/_array.cxx.o
[ 87%] Linking CXX shared module release/_array.so
[ 87%] Built target _array
Scanning dependencies of target _io_pyx
[ 91%] Compiling Cython CXX source for _io...
[ 91%] Built target _io_pyx
Scanning dependencies of target _io
[ 95%] Building CXX object CMakeFiles/_io.dir/_io.cxx.o
[100%] Linking CXX shared module release/_io.so
[100%] Built target _io
('Moving built C-extension', 'release/_array.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_array.so')
('Moving built C-extension', 'release/_config.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_config.so')
('Moving built C-extension', 'release/_error.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_error.so')
('Moving built C-extension', 'release/_io.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_io.so')
('Moving built C-extension', 'release/_jemalloc.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_jemalloc.so')
('Moving built C-extension', 'release/_memory.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_memory.so')
('Moving built C-extension', 'release/_parquet.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_parquet.so')
('Moving built C-extension', 'release/_table.so', 'to build path',
'/home/derdo/repos/arrow/python/pyarrow/_table.so')
(pyarrow-dev) derdo@prompt:~/repos/arrow/python$ py.test pyarrow
=========================== test session starts ===========================
platform linux2 -- Python 2.7.13, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/derdo/repos/arrow/python, inifile:
collected 210 items
pyarrow/tests/test_array.py ...........
pyarrow/tests/test_convert_builtin.py ......................
pyarrow/tests/test_convert_pandas.py ..........................x....
pyarrow/tests/test_feather.py ......................x....
pyarrow/tests/test_hdfs.py sssssssssssssss
pyarrow/tests/test_io.py ..................
pyarrow/tests/test_ipc.py ..........
pyarrow/tests/test_jemalloc.py ..
pyarrow/tests/test_parquet.py ......................
pyarrow/tests/test_scalars.py ..........
pyarrow/tests/test_schema.py .............
pyarrow/tests/test_table.py .............
pyarrow/tests/test_tensor.py ................
============ 193 passed, 15 skipped, 2 xfailed in 1.27 seconds ============
> [Docs] Guide for building Python from source on Ubuntu 14.04 LTS without conda
> ------------------------------------------------------------------------------
>
> Key: ARROW-955
> URL: https://issues.apache.org/jira/browse/ARROW-955
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Environment: Ubuntu - 3.19.0-80-generic #88~14.04.1-Ubuntu
> Python 2.7.6
> Reporter: Devang Shah
>
> I built pyarrow, arrow, and parquet-cpp from source - so that I could use the
> new read_row_group() interface and in general, have access to the latest
> versions. I ran into many issues during the build but was ultimately
> successful (notes below). However, I am not able to import pyarrow.parquet
> due to the following issue:
> >>import pyarrow.parquet
> Traceback (most recent call last):
> File "", line 1, in
> File "pyarrow/init.py", line 28, in
> import pyarrow._config
> ImportError: No module named _config
> This is similar to an issue reported in github/conda-forge/pyarrow-feedstock,
> where also I posted this...but I think this forum is more direct and
> appropriate - so re-posting here.
> I used instructions at https://arrow.apache.org/docs/python/install.html to
> build arrow/cpp, parquet-cpp, and then pyarrow, with the following deviations
> (I view them as possibly bugs in the instructions):
> arrow/cpp build:
> export ARROW_HOME=$HOME/local
> I had to specify -DARROW_PYTHON=on and -DPARQUET_ARROW=ON to the cmake
> command (besides the -DCMAKE_INSTALL_PREFIX=$ARROW_HOME)
> parquet-cpp build:
> export ARROW_HOME=$HOME/local
> cmake -DARROW_HOME=$HOME/local -DPARQUET_ARROW_LINKAGE=static
> -DPARQUET_ARROW=ON .
> make
> sudo make install ----> this installs parquet libs in the std systems
> location (/usr/local/lib) so that the pyarrow build (see below) can find the
> parquet libs
> pyarrow build:
> export ARROW_HOME=$HOME/local (not a deviation; just repeating here)
> export LD_LIBRARY_PATH=$HOME/local/lib:$HOME/parquet4/parquet-cpp/build/latest
> sudo python setup.py build_ext --with-parquet --with-jemalloc
> --build-type=release install
> sudo python setup.py install
> (sudo is needed to install in /usr/local/lib/python2.7/dist-packages )
> These are the steps and modifications to the instructions needed for me to
> build the pyarrow.parquet package. However, when I now try to import the
> package I get the error specified above.
> Maybe I did something wrong in my steps which I kind of put together by
> searching for these issues...but really can't tell what. It took me almost a
> whole day to get to the point where I can build pyarrow and parquet, and now
> I can't use what I built.
> Any comments, help appreciated! Thanks in advance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
