nealrichardson commented on PR #12849:
URL: https://github.com/apache/arrow/pull/12849#issuecomment-1098523529
What about a different approach? We don't need to use the download script
itself to download the files, but we do need to use versions.txt. We could use
a slightly different bash script to generate R syntax that would download the
files, then use R to download--no wget or curl or other system things required
beyond what R already has, just need bash.
To illustrate, I patched the existing download script:
```
diff --git a/cpp/thirdparty/download_dependencies.sh
b/cpp/thirdparty/download_dependencies.sh
index 7ffffa08c..2c0447cdb 100755
--- a/cpp/thirdparty/download_dependencies.sh
+++ b/cpp/thirdparty/download_dependencies.sh
@@ -30,14 +30,11 @@ else
DESTDIR=$1
fi
-DESTDIR=$(readlink -f "${DESTDIR}")
-
download_dependency() {
local url=$1
local out=$2
- wget --quiet --continue --output-document="${out}" "${url}" || \
- (echo "Failed downloading ${url}" 1>&2; exit 1)
+ echo 'download.file("'${url}'", "'${out}'")'
}
main() {
@@ -46,7 +43,6 @@ main() {
# Load `DEPENDENCIES` variable.
source ${SOURCE_DIR}/versions.txt
- echo "# Environment variables for offline Arrow build"
for ((i = 0; i < ${#DEPENDENCIES[@]}; i++)); do
local dep_packed=${DEPENDENCIES[$i]}
@@ -55,8 +51,6 @@ main() {
local out=${DESTDIR}/${dep_tar_name}
download_dependency "${dep_url}" "${out}"
-
- echo "export ${dep_url_var}=${out}"
done
}
```
in R we can get:
```
> system("source ../cpp/thirdparty/download_dependencies.sh /tmp",
intern=TRUE)
[1]
"download.file(\"https://github.com/abseil/abseil-cpp/archive/20210324.2.tar.gz\";,
\"/tmp/absl-20210324.2.tar.gz\")"
[2]
"download.file(\"https://github.com/aws/aws-sdk-cpp/archive/1.8.133.tar.gz\";,
\"/tmp/aws-sdk-cpp-1.8.133.tar.gz\")"
[3]
"download.file(\"https://github.com/awslabs/aws-checksums/archive/v0.1.12.tar.gz\";,
\"/tmp/aws-checksums-v0.1.12.tar.gz\")"
[4]
"download.file(\"https://github.com/awslabs/aws-c-common/archive/v0.6.9.tar.gz\";,
\"/tmp/aws-c-common-v0.6.9.tar.gz\")"
...
```
which you could then source.
I'm also ok with just fixing the one `readlink` line and saying that the
script requires bash and wget, possibly just adding that to the error message
that happens if the download script errors. That's much simpler. I don't think
R code and tests for whether we can determine if the current machine has bash
and wget installed are worth the trouble.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
