This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d264ee37f31 [SPARK-44688][INFRA] Add a file existence check before
executing `free_disk_space` and `free_disk_space_container`
d264ee37f31 is described below
commit d264ee37f316b32bf37fff770b72b5841b84f7cc
Author: yangjie01 <[email protected]>
AuthorDate: Sun Aug 6 17:41:11 2023 +0800
[SPARK-44688][INFRA] Add a file existence check before executing
`free_disk_space` and `free_disk_space_container`
### What changes were proposed in this pull request?
This pr add a file existence check before executing `dev/free_disk_space`
and `dev/free_disk_space_container`
### Why are the changes needed?
We added `free_disk_space` and `free_disk_space_container` to clean up the
disk, but because the daily tests of other branches and the master branch share
the yml file, we should check if the file exists before execution, otherwise it
will affect the daily tests of other branches.
- branch-3.5: https://github.com/apache/spark/actions/runs/5761479443
- branch-3.4: https://github.com/apache/spark/actions/runs/5760423900
- branch-3.3: https://github.com/apache/spark/actions/runs/5759384052
<img width="1073" alt="image"
src="https://github.com/apache/spark/assets/1475305/6e46b34b-645a-4da5-b9c3-8a89bfacabcb";>
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Action
Closes #42359 from LuciferYang/test-free_disk_space-exist.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.github/workflows/build_and_test.yml | 19 +++++++++++++++----
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 04585481a9c..cd68c0904d9 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -241,7 +241,10 @@ jobs:
restore-keys: |
${{ matrix.java }}-${{ matrix.hadoop }}-coursier-
- name: Free up disk space
- run: ./dev/free_disk_space
+ run: |
+ if [ -f ./dev/free_disk_space ]; then
+ ./dev/free_disk_space
+ fi
- name: Install Java ${{ matrix.java }}
uses: actions/setup-java@v3
with:
@@ -419,7 +422,9 @@ jobs:
# uninstall libraries dedicated for ML testing
python3.9 -m pip uninstall -y torch torchvision torcheval torchtnt
tensorboard mlflow
fi
- ./dev/free_disk_space_container
+ if [ -f ./dev/free_disk_space_container ]; then
+ ./dev/free_disk_space_container
+ fi
- name: Install Java ${{ matrix.java }}
uses: actions/setup-java@v3
with:
@@ -519,7 +524,10 @@ jobs:
restore-keys: |
sparkr-coursier-
- name: Free up disk space
- run: ./dev/free_disk_space_container
+ run: |
+ if [ -f ./dev/free_disk_space_container ]; then
+ ./dev/free_disk_space_container
+ fi
- name: Install Java ${{ inputs.java }}
uses: actions/setup-java@v3
with:
@@ -629,7 +637,10 @@ jobs:
restore-keys: |
docs-maven-
- name: Free up disk space
- run: ./dev/free_disk_space_container
+ run: |
+ if [ -f ./dev/free_disk_space_container ]; then
+ ./dev/free_disk_space_container
+ fi
- name: Install Java 8
uses: actions/setup-java@v3
with:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
