Gregory Hayes created ARROW-4935:
------------------------------------
Summary: Errors from jemalloc when building pyarrow from source on
OSX and Debian
Key: ARROW-4935
URL: https://issues.apache.org/jira/browse/ARROW-4935
Project: Apache Arrow
Issue Type: Bug
Components: C++, Python
Affects Versions: 0.12.1
Environment: OSX, Debian, Python==3.6.7
Reporter: Gregory Hayes
My attempts to build pyarrow from source are failing. I've set up the conda
environment using the instructions provided in the Develop instructions, and
have tried this on both Debian and OSX. When I run CMAKE in debug mode on OSX,
the output is:
{code:java}
-- Building using CMake version: 3.14.0
-- Arrow version: 0.13.0 (full: '0.13.0-SNAPSHOT')
-- clang-tidy not found
-- clang-format not found
-- infer found at /usr/local/bin/infer
-- Using ccache: /usr/local/bin/ccache
-- Found cpplint executable at
/Users/Greg/documents/repos/arrow/cpp/build-support/cpplint.py
-- Compiler command: env LANG=C
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-v
-- Compiler version: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
InstalledDir:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
-- Compiler id: AppleClang
Selected compiler clang 4.1.0svn
-- Arrow build warning level: CHECKIN
Configured for DEBUG build (set with cmake
-DCMAKE_BUILD_TYPE={release,debug,...})
-- Build Type: DEBUG
-- BOOST_VERSION: 1.67.0
-- BROTLI_VERSION: v0.6.0
-- CARES_VERSION: 1.15.0
-- DOUBLE_CONVERSION_VERSION: v3.1.1
-- FLATBUFFERS_VERSION: v1.10.0
-- GBENCHMARK_VERSION: v1.4.1
-- GFLAGS_VERSION: v2.2.0
-- GLOG_VERSION: v0.3.5
-- GRPC_VERSION: v1.18.0
-- GTEST_VERSION: 1.8.1
-- JEMALLOC_VERSION: 17c897976c60b0e6e4f4a365c751027244dada7a
-- LZ4_VERSION: v1.8.3
-- ORC_VERSION: 1.5.4
-- PROTOBUF_VERSION: v3.6.1
-- RAPIDJSON_VERSION: v1.1.0
-- RE2_VERSION: 2018-10-01
-- SNAPPY_VERSION: 1.1.3
-- THRIFT_VERSION: 0.11.0
-- ZLIB_VERSION: 1.2.8
-- ZSTD_VERSION: v1.3.7
-- Boost version: 1.68.0
-- Found the following Boost libraries:
-- regex
-- system
-- filesystem
-- Boost include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
-- Boost libraries:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_regex.dylib/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_system.dylib/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_filesystem.dylib
Added shared library dependency boost_system_shared:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_system.dylib
Added shared library dependency boost_filesystem_shared:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_filesystem.dylib
Added shared library dependency boost_regex_shared:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_regex.dylib
Added static library dependency double-conversion_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libdouble-conversion.a
-- double-conversion include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
-- double-conversion static library:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libdouble-conversion.a
-- GFLAGS_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
-- GFlags include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
-- GFlags static library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libgflags.a
Added static library dependency gflags_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libgflags.a
-- RapidJSON include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
-- Found the Flatbuffers library:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libflatbuffers.a
-- Flatbuffers include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
-- Flatbuffers compiler: /Users/Greg/anaconda3/envs/pyarrow-dev/bin/flatc
Added static library dependency jemalloc_static:
/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a
Added shared library dependency jemalloc_shared:
/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc.dylib
-- Found hdfs.h at:
/Users/Greg/documents/repos/arrow/cpp/thirdparty/hadoop/include/hdfs.h
-- Found the ZLIB shared library:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libz.dylib
Added shared library dependency zlib_shared:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libz.dylib
-- SNAPPY_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
-- Found the Snappy library:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libsnappy.dylib
Added static library dependency snappy_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libsnappy.a
-- Found the Brotli library:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlienc-static.a;/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlidec-static.a;/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlicommon-static.a
Added static library dependency brotli_enc_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlienc-static.a
Added static library dependency brotli_dec_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlidec-static.a
Added static library dependency brotli_common_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlicommon-static.a
Added static library dependency lz4_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/liblz4.a
Added static library dependency zstd_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libzstd.a
-- THRIFT_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
-- Thrift version: 0.12.0
-- Thrift include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
-- Thrift static library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libthrift.a
-- Thrift compiler: /Users/Greg/anaconda3/envs/pyarrow-dev/bin/thrift
-- GLOG_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
-- Found the GLog library:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libglog.dylib
-- Glog include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
-- Glog static library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libglog.a
Added static library dependency glog_static:
/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libglog.a
-- CMAKE_C_FLAGS: -Qunused-arguments -ggdb -O0 -Weverything
-Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-deprecated -Wno-weak-vtables
-Wno-padded -Wno-comma -Wno-unused-macros -Wno-unused-parameter
-Wno-unused-template -Wno-undef -Wno-shadow -Wno-switch-enum
-Wno-exit-time-destructors -Wno-global-constructors -Wno-weak-template-vtables
-Wno-undefined-reinterpret-cast -Wno-implicit-fallthrough
-Wno-unreachable-code-return -Wno-float-equal -Wno-missing-prototypes
-Wno-documentation-unknown-command -Wno-old-style-cast
-Wno-covered-switch-default -Wno-cast-align -Wno-vla-extension
-Wno-shift-sign-overflow -Wno-used-but-marked-unused
-Wno-missing-variable-declarations -Wno-gnu-zero-variadic-macro-arguments
-Wconversion -Wno-sign-conversion -Wno-disabled-macro-expansion
-Wno-format-nonliteral -Wno-missing-noreturn -Wno-gnu-folding-constant
-Wno-reserved-id-macro -Wno-range-loop-analysis -Wno-double-promotion
-Wno-undefined-func-template -Wno-zero-as-null-pointer-constant
-Wno-unknown-warning-option -Werror -Wno-unknown-warning-option -msse4.2
-maltivec -stdlib=libc++
-- CMAKE_CXX_FLAGS: -Qunused-arguments -ggdb -O0 -Weverything
-Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-deprecated -Wno-weak-vtables
-Wno-padded -Wno-comma -Wno-unused-macros -Wno-unused-parameter
-Wno-unused-template -Wno-undef -Wno-shadow -Wno-switch-enum
-Wno-exit-time-destructors -Wno-global-constructors -Wno-weak-template-vtables
-Wno-undefined-reinterpret-cast -Wno-implicit-fallthrough
-Wno-unreachable-code-return -Wno-float-equal -Wno-missing-prototypes
-Wno-documentation-unknown-command -Wno-old-style-cast
-Wno-covered-switch-default -Wno-cast-align -Wno-vla-extension
-Wno-shift-sign-overflow -Wno-used-but-marked-unused
-Wno-missing-variable-declarations -Wno-gnu-zero-variadic-macro-arguments
-Wconversion -Wno-sign-conversion -Wno-disabled-macro-expansion
-Wno-format-nonliteral -Wno-missing-noreturn -Wno-gnu-folding-constant
-Wno-reserved-id-macro -Wno-range-loop-analysis -Wno-double-promotion
-Wno-undefined-func-template -Wno-zero-as-null-pointer-constant
-Wno-unknown-warning-option -Werror -Wno-unknown-warning-option -msse4.2
-maltivec -stdlib=libc++
-- Configuring done
CMake Warning (dev):
Policy CMP0068 is not set: RPATH settings on macOS do not affect
install_name. Run "cmake --help-policy CMP0068" for policy details. Use
the cmake_policy command to set the policy and suppress this warning.
For compatibility with older versions of CMake, the install_name fields for
the following targets are still affected by RPATH settings:
arrow_python_shared
arrow_shared
parquet_shared
plasma_shared
This warning is for project developers. Use -Wno-dev to suppress it.
-- Generating done
-- Build files have been written to:
/Users/Greg/documents/repos/arrow/cpp/build{code}
After I run make -j4, the output is:
{code:java}
0%] Built target metadata_fbs
[ 1%] Performing configure step for 'jemalloc_ep'
[ 2%] Built target parquet-thrift-deps
[ 12%] Built target arrow_python_objlib
[ 19%] Built target plasma_objlib
[ 19%] Built target arrow_python_static
CMake Error at
/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep-stamp/jemalloc_ep-configure-DEBUG.cmake:49
(message):
Command failed: 1
'./autogen.sh'
'AR=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar'
'CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc'
'--prefix=/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist/'
'--with-jemalloc-prefix=je_arrow_'
'--with-private-namespace=je_arrow_private_' '--disable-tls'
See also
/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep-stamp/jemalloc_ep-configure-*.log
make[2]: *** [jemalloc_ep-prefix/src/jemalloc_ep-stamp/jemalloc_ep-configure]
Error 1
make[1]: *** [CMakeFiles/jemalloc_ep.dir/all] Error 2
make: *** [all] Error 2
{code}
And the output of the jemalloc log file is:
{code:java}
configure: error: in
`/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
autoconf
./configure --enable-autogen
AR=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar
CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
--prefix=/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist/
--with-jemalloc-prefix=je_arrow_ --with-private-namespace=je_arrow_private_
--disable-tls
checking for xsltproc... /usr/bin/xsltproc
checking for gcc...
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... Error 0 in ./configure
{code}
Any help / guidance I can get is appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
