This is an automated email from the ASF dual-hosted git repository.
philo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 21a41c08f [VL][CI] Use pre-installed celeborn to avoid download
failure (#5082)
21a41c08f is described below
commit 21a41c08ff28bb074aef03d6c024f2d19c0ab54f
Author: PHILO-HE <[email protected]>
AuthorDate: Fri Mar 22 20:33:14 2024 +0800
[VL][CI] Use pre-installed celeborn to avoid download failure (#5082)
---
.github/workflows/velox_be.yml | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/velox_be.yml b/.github/workflows/velox_be.yml
index b53c678e5..85db767b9 100644
--- a/.github/workflows/velox_be.yml
+++ b/.github/workflows/velox_be.yml
@@ -453,8 +453,8 @@ jobs:
- name: TPC-H SF1.0 && TPC-DS SF10.0 Parquet local spark3.2 with
Celeborn 0.4.0
run: |
$PATH_TO_GLUTEN_TE/$OS_IMAGE_NAME/gha/gha-checkout/exec.sh \
- 'wget
https://archive.apache.org/dist/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-bin.tgz
&& \
- tar xzf apache-celeborn-0.4.0-incubating-bin.tgz -C /opt/ && mv
/opt/apache-celeborn-0.4.0-incubating-bin /opt/celeborn && cd /opt/celeborn && \
+ 'cd /opt && mkdir -p celeborn && \
+ tar xzf apache-celeborn-0.4.0-incubating-bin.tgz -C /opt/celeborn
--strip-components=1 && cd celeborn && \
mv ./conf/celeborn-env.sh.template ./conf/celeborn-env.sh && \
echo -e
"CELEBORN_MASTER_MEMORY=4g\nCELEBORN_WORKER_MEMORY=4g\nCELEBORN_WORKER_OFFHEAP_MEMORY=8g"
> ./conf/celeborn-env.sh && \
echo -e "celeborn.worker.commitFiles.threads
128\nceleborn.worker.sortPartition.threads 64" > ./conf/celeborn-defaults.conf \
@@ -469,8 +469,8 @@ jobs:
- name: TPC-H SF1.0 && TPC-DS SF10.0 Parquet local spark3.2 with
Celeborn 0.3.2
run: |
$PATH_TO_GLUTEN_TE/$OS_IMAGE_NAME/gha/gha-checkout/exec.sh \
- 'wget
https://archive.apache.org/dist/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-bin.tgz
&& \
- tar xzf apache-celeborn-0.3.2-incubating-bin.tgz -C /opt/ && mv
/opt/apache-celeborn-0.3.2-incubating-bin /opt/celeborn && cd /opt/celeborn && \
+ 'cd /opt && mkdir -p celeborn && \
+ tar xzf apache-celeborn-0.3.2-incubating-bin.tgz -C /opt/celeborn
--strip-components=1 && cd celeborn && \
mv ./conf/celeborn-env.sh.template ./conf/celeborn-env.sh && \
echo -e
"CELEBORN_MASTER_MEMORY=4g\nCELEBORN_WORKER_MEMORY=4g\nCELEBORN_WORKER_OFFHEAP_MEMORY=8g"
> ./conf/celeborn-env.sh && \
echo -e "celeborn.worker.commitFiles.threads
128\nceleborn.worker.sortPartition.threads 64" > ./conf/celeborn-defaults.conf \
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
