[
https://issues.apache.org/jira/browse/ARROW-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joris Van den Bossche updated ARROW-7154:
-----------------------------------------
Description:
Since the docker-compose PR landed, I am having build errors like:
{code:java}
[361/376] Linking CXX executable debug/arrow-python-test
FAILED: debug/arrow-python-test
: && /home/joris/miniconda3/envs/arrow-dev/bin/ccache
/home/joris/miniconda3/envs/arrow-dev/bin/x86_64-conda_cos6-linux-gnu-c++
-Wno-noexcept-type -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong
-fno-plt -O2 -ffunction-sections -pipe -fdiagnostics-color=always -ggdb -O0
-Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror
-msse4.2 -g -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
-Wl,--disable-new-dtags -Wl,--gc-sections -rdynamic
src/arrow/python/CMakeFiles/arrow-python-test.dir/python_test.cc.o -o
debug/arrow-python-test
-Wl,-rpath,/home/joris/scipy/repos/arrow/cpp/build/debug:/home/joris/miniconda3/envs/arrow-dev/lib
debug/libarrow_python_test_main.a debug/libarrow_python.so.100.0.0
debug/libarrow_testing.so.100.0.0 debug/libarrow.so.100.0.0
/home/joris/miniconda3/envs/arrow-dev/lib/libpython3.7m.so -lpthread -lpthread
-ldl -lutil -lrt -ldl
/home/joris/miniconda3/envs/arrow-dev/lib/libdouble-conversion.a
/home/joris/miniconda3/envs/arrow-dev/lib/libglog.so
jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a -lrt
/home/joris/miniconda3/envs/arrow-dev/lib/libgtest.so -pthread && :
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
warning: libboost_filesystem.so.1.68.0, needed by debug/libarrow.so.100.0.0,
not found (try using -rpath or -rpath-link)
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
warning: libboost_system.so.1.68.0, needed by debug/libarrow.so.100.0.0, not
found (try using -rpath or -rpath-link)
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
debug/libarrow.so.100.0.0: undefined reference to
`boost::system::detail::generic_category_ncx()'
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
debug/libarrow.so.100.0.0: undefined reference to
`boost::filesystem::path::operator/=(boost::filesystem::path const&)'
collect2: error: ld returned 1 exit status
{code}
which contains warnings like "warning: libboost_filesystem.so.1.68.0, needed by
debug/libarrow.so.100.0.0, not found" (although this is certainly present).
The error is triggered by having {{-DARROW_BUILD_TESTS=ON}}. If that is set to
OFF, it works fine.
It also seems to be related to this specific change in the docker compose PR:
{code:java}
diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
index c80ac3310..3b3c9eb8f 100644
--- a/cpp/CMakeLists.txt
+++ b/cpp/CMakeLists.txt
@@ -266,6 +266,15 @@ endif(UNIX)
# Set up various options
#
-if(ARROW_BUILD_TESTS OR ARROW_BUILD_BENCHMARKS)
- # Currently the compression tests require at least these libraries; bz2 and
- # zstd are optional. See ARROW-3984
- set(ARROW_WITH_BROTLI ON)
- set(ARROW_WITH_LZ4 ON)
- set(ARROW_WITH_SNAPPY ON)
- set(ARROW_WITH_ZLIB ON)
-endif()
-
if(ARROW_BUILD_TESTS OR ARROW_BUILD_INTEGRATION)
set(ARROW_JSON ON)
endif()
{code}
If I add that back, the build works.
With only `set(ARROW_WITH_BROTLI ON)`, it still fails
With only `set(ARROW_WITH_LZ4 ON)`, it also fails but with an error about
liblz4 instead of libboost (but also liblz4 is actually present)
With only `set(ARROW_WITH_SNAPPY ON)`, it works
With only `set(ARROW_WITH_ZLIB ON)`, it also fails but with an error about
libz.so.1 not found
With both `set(ARROW_WITH_SNAPPY ON)` and `set(ARROW_WITH_ZLIB ON)`, it also
works. So it seems that the absence of snappy causes others to fail.
In the recommended build settings in the development docs
([https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst#build-and-test),]
the compression libraries are enabled. But I was still building without them
(stemming from the time they were enabled by default). So I was using:
{code}
cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME -GNinja \
-DCMAKE_INSTALL_LIBDIR=lib \
-DARROW_PARQUET=ON \
-DARROW_PYTHON=ON \
-DARROW_BUILD_TESTS=ON \
..
{code}
was:
Since the docker-compose PR landed, I am having build errors like:
{code:java}
[361/376] Linking CXX executable debug/arrow-python-test
FAILED: debug/arrow-python-test
: && /home/joris/miniconda3/envs/arrow-dev/bin/ccache
/home/joris/miniconda3/envs/arrow-dev/bin/x86_64-conda_cos6-linux-gnu-c++
-Wno-noexcept-type -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong
-fno-plt -O2 -ffunction-sections -pipe -fdiagnostics-color=always -ggdb -O0
-Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror
-msse4.2 -g -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
-Wl,--disable-new-dtags -Wl,--gc-sections -rdynamic
src/arrow/python/CMakeFiles/arrow-python-test.dir/python_test.cc.o -o
debug/arrow-python-test
-Wl,-rpath,/home/joris/scipy/repos/arrow/cpp/build/debug:/home/joris/miniconda3/envs/arrow-dev/lib
debug/libarrow_python_test_main.a debug/libarrow_python.so.100.0.0
debug/libarrow_testing.so.100.0.0 debug/libarrow.so.100.0.0
/home/joris/miniconda3/envs/arrow-dev/lib/libpython3.7m.so -lpthread -lpthread
-ldl -lutil -lrt -ldl
/home/joris/miniconda3/envs/arrow-dev/lib/libdouble-conversion.a
/home/joris/miniconda3/envs/arrow-dev/lib/libglog.so
jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a -lrt
/home/joris/miniconda3/envs/arrow-dev/lib/libgtest.so -pthread && :
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
warning: libboost_filesystem.so.1.68.0, needed by debug/libarrow.so.100.0.0,
not found (try using -rpath or -rpath-link)
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
warning: libboost_system.so.1.68.0, needed by debug/libarrow.so.100.0.0, not
found (try using -rpath or -rpath-link)
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
debug/libarrow.so.100.0.0: undefined reference to
`boost::system::detail::generic_category_ncx()'
/home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
debug/libarrow.so.100.0.0: undefined reference to
`boost::filesystem::path::operator/=(boost::filesystem::path const&)'
collect2: error: ld returned 1 exit status
{code}
which contains warnings like "warning: libboost_filesystem.so.1.68.0, needed by
debug/libarrow.so.100.0.0, not found" (although this is certainly present).
The error is triggered by having {{-DARROW_BUILD_TESTS=ON}}. If that is set to
OFF, it works fine.
It also seems to be related to this specific change in the docker compose PR:
{code:java}
diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
index c80ac3310..3b3c9eb8f 100644
--- a/cpp/CMakeLists.txt
+++ b/cpp/CMakeLists.txt
@@ -266,6 +266,15 @@ endif(UNIX)
# Set up various options
#
-if(ARROW_BUILD_TESTS OR ARROW_BUILD_BENCHMARKS)
- # Currently the compression tests require at least these libraries; bz2 and
- # zstd are optional. See ARROW-3984
- set(ARROW_WITH_BROTLI ON)
- set(ARROW_WITH_LZ4 ON)
- set(ARROW_WITH_SNAPPY ON)
- set(ARROW_WITH_ZLIB ON)
-endif()
-
if(ARROW_BUILD_TESTS OR ARROW_BUILD_INTEGRATION)
set(ARROW_JSON ON)
endif()
{code}
If I add that back, the build works.
With only `set(ARROW_WITH_BROTLI ON)`, it still fails
With only `set(ARROW_WITH_LZ4 ON)`, it also fails but with an error about
liblz4 instead of libboost (but also liblz4 is actually present)
With only `set(ARROW_WITH_SNAPPY ON)`, it works
With only `set(ARROW_WITH_ZLIB ON)`, it also fails but with an error about
libz.so.1 not found
So it seems that the absence of snappy causes others to fail.
In the recommended build settings in the development docs
([https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst#build-and-test),]
the compression libraries are enabled. But I was still building without them
(stemming from the time they were enabled by default). So I was using:
{code}
cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME -GNinja \
-DCMAKE_INSTALL_LIBDIR=lib \
-DARROW_PARQUET=ON \
-DARROW_PYTHON=ON \
-DARROW_BUILD_TESTS=ON \
..
{code}
> [C++] Build error when building tests but not with snappy
> ---------------------------------------------------------
>
> Key: ARROW-7154
> URL: https://issues.apache.org/jira/browse/ARROW-7154
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Joris Van den Bossche
> Priority: Major
>
> Since the docker-compose PR landed, I am having build errors like:
> {code:java}
> [361/376] Linking CXX executable debug/arrow-python-test
> FAILED: debug/arrow-python-test
> : && /home/joris/miniconda3/envs/arrow-dev/bin/ccache
> /home/joris/miniconda3/envs/arrow-dev/bin/x86_64-conda_cos6-linux-gnu-c++
> -Wno-noexcept-type -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0
> -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong
> -fno-plt -O2 -ffunction-sections -pipe -fdiagnostics-color=always -ggdb -O0
> -Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror
> -msse4.2 -g -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro
> -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -rdynamic
> src/arrow/python/CMakeFiles/arrow-python-test.dir/python_test.cc.o -o
> debug/arrow-python-test
> -Wl,-rpath,/home/joris/scipy/repos/arrow/cpp/build/debug:/home/joris/miniconda3/envs/arrow-dev/lib
> debug/libarrow_python_test_main.a debug/libarrow_python.so.100.0.0
> debug/libarrow_testing.so.100.0.0 debug/libarrow.so.100.0.0
> /home/joris/miniconda3/envs/arrow-dev/lib/libpython3.7m.so -lpthread
> -lpthread -ldl -lutil -lrt -ldl
> /home/joris/miniconda3/envs/arrow-dev/lib/libdouble-conversion.a
> /home/joris/miniconda3/envs/arrow-dev/lib/libglog.so
> jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a -lrt
> /home/joris/miniconda3/envs/arrow-dev/lib/libgtest.so -pthread && :
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
> warning: libboost_filesystem.so.1.68.0, needed by debug/libarrow.so.100.0.0,
> not found (try using -rpath or -rpath-link)
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
> warning: libboost_system.so.1.68.0, needed by debug/libarrow.so.100.0.0, not
> found (try using -rpath or -rpath-link)
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
> debug/libarrow.so.100.0.0: undefined reference to
> `boost::system::detail::generic_category_ncx()'
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld:
> debug/libarrow.so.100.0.0: undefined reference to
> `boost::filesystem::path::operator/=(boost::filesystem::path const&)'
> collect2: error: ld returned 1 exit status
> {code}
> which contains warnings like "warning: libboost_filesystem.so.1.68.0, needed
> by debug/libarrow.so.100.0.0, not found" (although this is certainly present).
> The error is triggered by having {{-DARROW_BUILD_TESTS=ON}}. If that is set
> to OFF, it works fine.
> It also seems to be related to this specific change in the docker compose PR:
> {code:java}
> diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
> index c80ac3310..3b3c9eb8f 100644
> --- a/cpp/CMakeLists.txt
> +++ b/cpp/CMakeLists.txt
> @@ -266,6 +266,15 @@ endif(UNIX)
> # Set up various options
> #
> -if(ARROW_BUILD_TESTS OR ARROW_BUILD_BENCHMARKS)
> - # Currently the compression tests require at least these libraries; bz2 and
> - # zstd are optional. See ARROW-3984
> - set(ARROW_WITH_BROTLI ON)
> - set(ARROW_WITH_LZ4 ON)
> - set(ARROW_WITH_SNAPPY ON)
> - set(ARROW_WITH_ZLIB ON)
> -endif()
> -
> if(ARROW_BUILD_TESTS OR ARROW_BUILD_INTEGRATION)
> set(ARROW_JSON ON)
> endif()
> {code}
> If I add that back, the build works.
> With only `set(ARROW_WITH_BROTLI ON)`, it still fails
> With only `set(ARROW_WITH_LZ4 ON)`, it also fails but with an error about
> liblz4 instead of libboost (but also liblz4 is actually present)
> With only `set(ARROW_WITH_SNAPPY ON)`, it works
> With only `set(ARROW_WITH_ZLIB ON)`, it also fails but with an error about
> libz.so.1 not found
> With both `set(ARROW_WITH_SNAPPY ON)` and `set(ARROW_WITH_ZLIB ON)`, it also
> works. So it seems that the absence of snappy causes others to fail.
> In the recommended build settings in the development docs
> ([https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst#build-and-test),]
> the compression libraries are enabled. But I was still building without them
> (stemming from the time they were enabled by default). So I was using:
> {code}
> cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME -GNinja \
> -DCMAKE_INSTALL_LIBDIR=lib \
> -DARROW_PARQUET=ON \
> -DARROW_PYTHON=ON \
> -DARROW_BUILD_TESTS=ON \
> ..
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
