This is an automated email from the ASF dual-hosted git repository.
philo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new d974db8dd [VL] Fix vcpkg binary caching in docker image (#7331)
d974db8dd is described below
commit d974db8dd5e3fe00a54008ef57725a854f1d59ca
Author: PHILO-HE <[email protected]>
AuthorDate: Tue Sep 24 17:05:40 2024 +0800
[VL] Fix vcpkg binary caching in docker image (#7331)
---
.github/workflows/build_bundle_package.yml | 3 +--
dev/vcpkg/README.md | 20 +++++---------------
dev/vcpkg/docker/Dockerfile.gha | 6 ++++--
3 files changed, 10 insertions(+), 19 deletions(-)
diff --git a/.github/workflows/build_bundle_package.yml
b/.github/workflows/build_bundle_package.yml
index cf70eeb1a..da250f15a 100644
--- a/.github/workflows/build_bundle_package.yml
+++ b/.github/workflows/build_bundle_package.yml
@@ -50,10 +50,9 @@ jobs:
cd $GITHUB_WORKSPACE/ep/build-velox/src && \
./get_velox.sh && \
source /opt/rh/devtoolset-9/enable && \
- source /opt/gluten//dev/vcpkg/env.sh && \
cd $GITHUB_WORKSPACE/ && \
export NUM_THREADS=4
- ./dev/builddeps-veloxbe.sh --build_tests=OFF --build_benchmarks=OFF
--enable_s3=OFF \
+ ./dev/builddeps-veloxbe.sh --enable_vcpkg=ON --build_tests=OFF
--build_benchmarks=OFF --enable_s3=OFF \
--enable_gcs=OFF --enable_hdfs=ON --enable_abfs=OFF
- name: Upload native libs
uses: actions/upload-artifact@v2
diff --git a/dev/vcpkg/README.md b/dev/vcpkg/README.md
index a5efef1e5..c5f1dc4a5 100644
--- a/dev/vcpkg/README.md
+++ b/dev/vcpkg/README.md
@@ -40,22 +40,12 @@ For unsupported linux distro, you can install the following
packages from packag
* openjdk 8
* maven
-### Build and setup thirdparty depends
+### Build gluten + velox with vcpkg installed dependencies
-Simply run:
+With `--enable_vcpkg=ON`, the below script will install all static libraries
into `./vcpkg_installed/`. And it will
+also set `$PATH` and `$CMAKE_TOOLCHAIN_FILE` to make CMake to locate the
binary tools and libraries.
+You can configure [binary
cache](https://learn.microsoft.com/en-us/vcpkg/users/binarycaching) to
accelerate the build.
``` sh
-source $GLUTEN_REPO/dev/vcpkg/env.sh
-```
-
-This script will install all static libraries into the `./vcpkg_installed/`
-directory and set the `$PATH` and `$CMAKE_TOOLCHAIN_FILE`.
-This make build systems to locate the binary tools and libraries.
-It will take about 15~30 minutes to download and build all dependencies from
source.
-You can configure [binary
cache](https://learn.microsoft.com/en-us/vcpkg/users/binarycaching) to
accelerate the next setup.
-
-### Build gluten + velox
-
-``` sh
-$GLUTEN_REPO/dev/buildbundle-veloxbe.sh --build_tests=ON --build_benchmarks=ON
--enable_s3=ON --enable_hdfs=ON
+$GLUTEN_REPO/dev/buildbundle-veloxbe.sh --enable_vcpkg=ON --build_tests=ON
--build_benchmarks=ON --enable_s3=ON --enable_hdfs=ON
```
\ No newline at end of file
diff --git a/dev/vcpkg/docker/Dockerfile.gha b/dev/vcpkg/docker/Dockerfile.gha
index d37c449cb..5fb164189 100644
--- a/dev/vcpkg/docker/Dockerfile.gha
+++ b/dev/vcpkg/docker/Dockerfile.gha
@@ -26,5 +26,7 @@ RUN mkdir -p /var/cache/vcpkg
ENV VCPKG_BINARY_SOURCES=clear;files,/var/cache/vcpkg,readwrite
# Build arrow, then install the native libs to system paths and jar package to
.m2/ directory.
-RUN cd /opt/gluten && source /opt/rh/devtoolset-9/enable && source
./dev/vcpkg/env.sh && \
- bash ./dev/builddeps-veloxbe.sh build_arrow && rm -rf /opt/gluten
+RUN cd /opt/gluten && source /opt/rh/devtoolset-9/enable && \
+ bash ./dev/builddeps-veloxbe.sh --enable_vcpkg=ON --build_tests=ON
--enable_s3=ON --enable_gcs=ON \
+ --enable_hdfs=ON --enable_abfs=ON
build_arrow && \
+ rm -rf /opt/gluten
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
