karldw commented on PR #12849:
URL: https://github.com/apache/arrow/pull/12849#issuecomment-1103860751
> Not immediately relevant here but in tools/nixlibs.R, there is code in
`set_thirdparty_urls()` that generates the env vars that point to the files,
and there are more cases that needs special handling now: google cloud cpp
(comes out as ARROW_GOOGLE_URL) and nhlomann json (comes out as
ARROW_NLOHMANN_URL). These aren't used now because we haven't added GCP
bindings to the R package yet. But whenever we do, we'll need to fix those urls
or else this fat package won't work for them.
One option might be to modify `download_dependencies_R.sh` to pull the
actual names from versions.txt? Here's a demo that spits out the names we need
in CSV format. (In reality we would want a command line flag rather than simply
removing the download code.)
```diff
@@ -44,20 +44,25 @@ download_dependency() {
echo 'download.file("'${url}'", "'${out}'", quiet = TRUE)'
}
+print_tar_name() {
+ local url_var=$1
+ local tar_name=$2
+ echo "'${url_var}','${tar_name}'"
+}
+
main() {
- mkdir -p "${DESTDIR}"
# Load `DEPENDENCIES` variable.
source ${SOURCE_DIR}/cpp/thirdparty/versions.txt
+ echo 'env_var,filename'
for ((i = 0; i < ${#DEPENDENCIES[@]}; i++)); do
local dep_packed=${DEPENDENCIES[$i]}
# Unpack each entry of the form "$home_var $tar_out $dep_url"
IFS=" " read -r dep_url_var dep_tar_name dep_url <<< "${dep_packed}"
- local out=${DESTDIR}/${dep_tar_name}
- download_dependency "${dep_url}" "${out}"
+ print_tar_name "${dep_url_var}" "${dep_tar_name}"
done
}
```
<details>
<summary>
Output
</summary>
```
env_var,filename
'ARROW_ABSL_URL','absl-20210324.2.tar.gz'
'ARROW_AWSSDK_URL','aws-sdk-cpp-1.8.133.tar.gz'
'ARROW_AWS_CHECKSUMS_URL','aws-checksums-v0.1.12.tar.gz'
'ARROW_AWS_C_COMMON_URL','aws-c-common-v0.6.9.tar.gz'
'ARROW_AWS_C_EVENT_STREAM_URL','aws-c-event-stream-v0.1.5.tar.gz'
'ARROW_BOOST_URL','boost-1.75.0.tar.gz'
'ARROW_BROTLI_URL','brotli-v1.0.9.tar.gz'
'ARROW_BZIP2_URL','bzip2-1.0.8.tar.gz'
'ARROW_CARES_URL','cares-1.17.2.tar.gz'
'ARROW_CRC32C_URL','crc32c-1.1.2.tar.gz'
'ARROW_GBENCHMARK_URL','gbenchmark-v1.6.0.tar.gz'
'ARROW_GFLAGS_URL','gflags-v2.2.2.tar.gz'
'ARROW_GLOG_URL','glog-v0.5.0.tar.gz'
'ARROW_GOOGLE_CLOUD_CPP_URL','google-cloud-cpp-v1.39.0.tar.gz'
'ARROW_GRPC_URL','grpc-v1.35.0.tar.gz'
'ARROW_GTEST_URL','gtest-1.11.0.tar.gz'
'ARROW_JEMALLOC_URL','jemalloc-5.2.1.tar.bz2'
'ARROW_LZ4_URL','lz4-8f61d8eb7c6979769a484cde8df61ff7c4c77765.tar.gz'
'ARROW_MIMALLOC_URL','mimalloc-v1.7.3.tar.gz'
'ARROW_NLOHMANN_JSON_URL','nlohmann-json-v3.10.2.tar.gz'
'ARROW_OPENTELEMETRY_URL','opentelemetry-cpp-v1.2.0.tar.gz'
'ARROW_OPENTELEMETRY_PROTO_URL','opentelemetry-proto-v0.11.0.tar.gz'
'ARROW_ORC_URL','orc-1.7.3.tar.gz'
'ARROW_PROTOBUF_URL','protobuf-v3.18.1.tar.gz'
'ARROW_RAPIDJSON_URL','rapidjson-1a803826f1197b5e30703afe4b9c0e7dd48074f5.tar.gz'
'ARROW_RE2_URL','re2-2021-11-01.tar.gz'
'ARROW_SNAPPY_URL','snappy-1.1.9.tar.gz'
'ARROW_THRIFT_URL','thrift-0.13.0.tar.gz'
'ARROW_UTF8PROC_URL','utf8proc-v2.7.0.tar.gz'
'ARROW_XSIMD_URL','xsimd-7d1778c3b38d63db7cec7145d939f40bc5d859d1.tar.gz'
'ARROW_ZLIB_URL','zlib-1.2.12.tar.gz'
'ARROW_ZSTD_URL','zstd-v1.5.1.tar.gz'
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
