jonkeane commented on code in PR #13483:
URL: https://github.com/apache/arrow/pull/13483#discussion_r989068873
##########
ci/scripts/r_revdepcheck.sh:
##########
@@ -19,11 +19,44 @@
set -ex
: ${R_BIN:=R}
-
+# When revdep runs with > 1 worker the checks for {targets} time out for
+# some reason.
+: ${ARROW_REVDEP_WORKERS:=1}
+# But we do want to use all cores while building arrow to speed up the
+# installation so this is used to set MAKEFLAGS
Review Comment:
😍
##########
docker-compose.yml:
##########
@@ -1373,21 +1373,20 @@ services:
# Usage:
# docker-compose build r-revdepcheck
# docker-compose run r-revdepcheck
- image: ${REPO}:r-rstudio-r-base-4.0-focal-revdepcheck
+ image: ${REPO}:r-rstudio-r-base-4.2-focal-revdepcheck
build:
context: .
dockerfile: ci/docker/linux-r.dockerfile
cache_from:
- - ${REPO}:r-rstudio-r-base-4.0-focal-revdepcheck
+ - ${REPO}:r-rstudio-r-base-4.2-focal-revdepcheck
args:
- base: rstudio/r-base:4.0-focal
+ base: rstudio/r-base:4.2-focal
r_dev: ${ARROW_R_DEV}
tz: ${TZ}
shm_size: *shm-size
environment:
- LIBARROW_DOWNLOAD: "true"
- LIBARROW_MINIMAL: "false"
- ARROW_SOURCE_HOME: "/arrow"
+ N_JOBS:
+ ARROW_REVDEP_WORKERS:
Review Comment:
Is the empty argument here valid? I don't think I've seen it before, but if
it works, that's fine...
##########
ci/scripts/r_revdepcheck.sh:
##########
@@ -19,11 +19,40 @@
set -ex
: ${R_BIN:=R}
-
+: ${ARROW_REVDEP_WORKERS:=1)}
+: ${N_JOBS:=$(nproc)}
source_dir=${1}/r
# cpp building dependencies
-apt install -y cmake
+apt update -y -q && \
+apt install -y \
+ cmake \
+ libbrotli-dev \
+ libbz2-dev \
+ libc-ares-dev \
+ libcurl4-openssl-dev \
+ libgflags-dev \
+ libgoogle-glog-dev \
+ liblz4-dev \
+ libprotobuf-dev \
+ libprotoc-dev \
+ libradospp-dev \
+ libre2-dev \
+ libsnappy-dev \
+ libssl-dev \
+ libthrift-dev \
+ libutf8proc-dev \
+ libzstd-dev \
+ nlohmann-json3-dev \
+ pkg-config \
+ protobuf-compiler \
+ python3-dev \
+ python3-pip \
+ python3-rados \
+ rados-objclass-dev \
+ rapidjson-dev \
+ tzdata \
+ wget
Review Comment:
Sure, maybe a TODO that mentions that we might want to consolidate it at
some point?
##########
ci/scripts/r_revdepcheck.sh:
##########
@@ -42,16 +75,26 @@ apt install -y libxml2-dev \
libgeos-dev \
libproj-dev
-pushd ${source_dir}
+
+# We have to be in source_dir so that cpp source detection works
+pushd $source_dir
printenv
+# copy over cpp source
+make sync-cpp
+
# By default, aws-sdk tries to contact a non-existing local ip host
# to retrieve metadata. Disable this so that S3FileSystem tests run faster.
export AWS_EC2_METADATA_DISABLED=TRUE
# Set crancache dir so we can cache it
-export CRANCACHE_DIR="/arrow/.crancache"
+export CRANCACHE_DIR="${1}/.crancache"
+
+# Don't use system boost to prevent issues with missing components
+export EXTRA_CMAKE_FLAGS='-DBoost_SOURCE=BUNDLED'
Review Comment:
This might be unfounded, but I have a small fear that adding extra flags
here means we're drifting away from how this is run on CRAN. This _probably_
doesn't matter. But is it possible to run this build without the extra flags?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
