[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637494#comment-17637494
]
Arjan van der Velde edited comment on ARROW-17374 at 11/22/22 10:57 PM:
------------------------------------------------------------------------
I have a conda environment file. I haven't stripped it down to the minimum
requirements for arrow but it illustrates the issue. The issue happens on a
linux (centos) install and not on macOS where it seems to be using imported
libraries from brew.
[^environment.yml]
{code:java}
# Set up conda environment and activate
conda env create -n testme -f environment.yml
conda activate testme
# Build required 'decor' which doesn't exist in any conda repo
R -e 'install.packages("decor", repos="http://cran.us.r-project.org";)'
# Build arrow
mkdir -p /tmp/git && \
cd /tmp/git && git clone 'https://github.com/apache/arrow' arrow &&
cd /tmp/git/arrow/r && \
ARROW_S3=OFF LIBARROW_MINIMAL=FALSE ARROW_R_DEV=TRUE R CMD INSTALL . {code}
The build fails with:
{noformat}
...._JSON -DARROW_R_WITH_GCS
-I'/home/username/miniconda3/envs/testme/lib/R/library/cpp11/include' -DNDEBUG
-D_FORTIFY_SOURCE=2 -O2 -isystem /home/username/miniconda3/envs/testme/include
-I/home/username/miniconda3/envs/testme/include
-Wl,-rpath-link,/home/username/miniconda3/envs/testme/lib -fpic
-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell
-ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2
-ffunction-sections -pipe -isystem
/home/username/miniconda3/envs/testme/include
-fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/r-base-split_1665472261643/work=/usr/local/src/conda/r-base-4.1.3
-fdebug-prefix-map=/home/username/miniconda3/envs/testme=/usr/local/src/conda-prefix
-c type_infer.cpp -o type_infer.ox86_64-conda-linux-gnu-c++ -std=gnu++17
-shared -L/home/username/miniconda3/envs/testme/lib/R/lib -Wl,-O2
-Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
-Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined
-Wl,-rpath,/home/username/miniconda3/envs/testme/lib
-Wl,-rpath-link,/home/username/miniconda3/envs/testme/lib
-L/home/username/miniconda3/envs/testme/lib -o arrow.so RTasks.o altrep.o
array.o array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o
chunkedarray.o compression.o compute-exec.o compute.o config.o csv.o dataset.o
datatype.o expression.o extension-impl.o feather.o field.o filesystem.o
imports.o io.o json.o memorypool.o message.o parquet.o r_to_arrow.o
recordbatch.o recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o
scalar.o schema.o symbols.o table.o threadpool.o type_infer.o
-L/tmp/git/arrow/r/libarrow/arrow-10.0.0.9000/lib -larrow_dataset -lparquet
-Llibarrow/arrow-10.0.0.9000/lib -L/home/username/miniconda3/envs/testme/lib
-larrow SNAPPY_LIB-NOTFOUND
/home/username/miniconda3/envs/testme/lib/libre2.so.9.0.0 /usr/lib64/libbz2.so
-pthread -larrow_bundled_dependencies -lbrotlidec -lbrotlienc -lbrotlicommon
-lthrift -lz -llz4 -lzstd -lutf8proc -larrow -larrow_bundled_dependencies
-larrow_dataset -lparquet -lssl -lcrypto -lcurl
-L/home/username/miniconda3/envs/testme/lib/R/lib -lR
/home/username/miniconda3/envs/testme/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld:
cannot find SNAPPY_LIB-NOTFOUND: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [/home/username/miniconda3/envs/testme/lib/R/share/make/shlib.mk:10:
arrow.so] Error 1
ERROR: compilation failed for package ‘arrow’
* removing ‘/home/username/miniconda3/envs/testme/lib/R/library/arrow’
* restoring previous
‘/home/username/miniconda3/envs/testme/lib/R/library/arrow’{noformat}
was (Author: JIRAUSER297783):
I have a conda environment file. I haven't stripped it down to the minimum
requirements for arrow but it illustrates the issue. The issue happens on a
linux (centos) install and not on macOS where it seems to be using imported
libraries from brew.
[^environment.yml]
{code:java}
# Set up conda environment and activate
conda env create -n testme -f environment.yml
conda activate testme
# Build required 'decor' which doesn't exist in any conda repo
R -e 'install.packages("decor", repos="http://cran.us.r-project.org";)'
# Build arrow
mkdir -p /tmp/git && \
cd /tmp/git && git clone 'https://github.com/apache/arrow' arrow &&
cd /tmp/git/arrow/r && \
ARROW_S3=OFF LIBARROW_MINIMAL=FALSE ARROW_R_DEV=TRUE R CMD INSTALL . {code}
The build fails with:
{noformat}
...._JSON -DARROW_R_WITH_GCS
-I'/home/username/miniconda3/envs/testme/lib/R/library/cpp11/include' -DNDEBUG
-D_FORTIFY_SOURCE=2 -O2 -isystem /home/username/miniconda3/envs/testme/include
-I/home/username/miniconda3/envs/testme/include
-Wl,-rpath-link,/home/username/miniconda3/envs/testme/lib -fpic
-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell
-ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2
-ffunction-sections -pipe -isystem
/home/username/miniconda3/envs/testme/include
-fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/r-base-split_1665472261643/work=/usr/local/src/conda/r-base-4.1.3
-fdebug-prefix-map=/home/username/miniconda3/envs/testme=/usr/local/src/conda-prefix
-c type_infer.cpp -o type_infer.ox86_64-conda-linux-gnu-c++ -std=gnu++17
-shared -L/home/username/miniconda3/envs/testme/lib/R/lib -Wl,-O2
-Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
-Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined
-Wl,-rpath,/home/username/miniconda3/envs/testme/lib
-Wl,-rpath-link,/home/username/miniconda3/envs/testme/lib
-L/home/username/miniconda3/envs/testme/lib -o arrow.so RTasks.o altrep.o
array.o array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o
chunkedarray.o compression.o compute-exec.o compute.o config.o csv.o dataset.o
datatype.o expression.o extension-impl.o feather.o field.o filesystem.o
imports.o io.o json.o memorypool.o message.o parquet.o r_to_arrow.o
recordbatch.o recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o
scalar.o schema.o symbols.o table.o threadpool.o type_infer.o
-L/tmp/git/arrow/r/libarrow/arrow-10.0.0.9000/lib -larrow_dataset -lparquet
-Llibarrow/arrow-10.0.0.9000/lib -L/home/username/miniconda3/envs/testme/lib
-larrow SNAPPY_LIB-NOTFOUND
/home/username/miniconda3/envs/testme/lib/libre2.so.9.0.0 /usr/lib64/libbz2.so
-pthread -larrow_bundled_dependencies -lbrotlidec -lbrotlienc -lbrotlicommon
-lthrift -lz -llz4 -lzstd -lutf8proc -larrow -larrow_bundled_dependencies
-larrow_dataset -lparquet -lssl -lcrypto -lcurl
-L/home/username/miniconda3/envs/testme/lib/R/lib
-lR/home/username/miniconda3/envs/testme/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld:
cannot find SNAPPY_LIB-NOTFOUND: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [/home/username/miniconda3/envs/testme/lib/R/share/make/shlib.mk:10:
arrow.so] Error 1
ERROR: compilation failed for package ‘arrow’
* removing ‘/home/username/miniconda3/envs/testme/lib/R/library/arrow’
* restoring previous
‘/home/username/miniconda3/envs/testme/lib/R/library/arrow’{noformat}
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --------------------------------------------------
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
> Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
> Reporter: Shane Brennan
> Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
