This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 015719ee95b1 [SPARK-47516][INFRA] Move `remove unused installation
package logic` from `each test job` to `create the docker image`
015719ee95b1 is described below
commit 015719ee95b1dfba232df799c54835526e3dadff
Author: panbingkun <[email protected]>
AuthorDate: Mon Mar 25 08:52:19 2024 +0900
[SPARK-47516][INFRA] Move `remove unused installation package logic` from
`each test job` to `create the docker image`
### What changes were proposed in this pull request?
The pr aims to move `remove unused installation package logic` from `each
test job` to `create the docker image`.
### Why are the changes needed?
Speed up the testing speed of GA once and for all.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45659 from panbingkun/improve_container_install_packages.
Authored-by: panbingkun <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
dev/free_disk_space_container | 10 ----------
dev/infra/Dockerfile | 8 ++++++++
2 files changed, 8 insertions(+), 10 deletions(-)
diff --git a/dev/free_disk_space_container b/dev/free_disk_space_container
index ac5ca9638479..c60125fa3909 100755
--- a/dev/free_disk_space_container
+++ b/dev/free_disk_space_container
@@ -31,14 +31,4 @@ rm -rf /__t/CodeQL
rm -rf /__t/go
rm -rf /__t/node
-apt-get remove --purge -y '^aspnet.*'
-apt-get remove --purge -y '^dotnet-.*'
-apt-get remove --purge -y '^llvm-.*'
-apt-get remove --purge -y 'php.*'
-apt-get remove --purge -y '^mongodb-.*'
-apt-get remove --purge -y snapd google-chrome-stable microsoft-edge-stable
firefox
-apt-get remove --purge -y azure-cli google-cloud-sdk mono-devel powershell
libgl1-mesa-dri
-apt-get autoremove --purge -y
-apt-get clean
-
df -h
diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index f17ee58c9d90..9e88ed794c21 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -135,3 +135,11 @@ RUN python3.12 -m pip install $BASIC_PIP_PKGS
$CONNECT_PIP_PKGS lxml && \
python3.12 -m pip install torch torchvision --index-url
https://download.pytorch.org/whl/cpu && \
python3.12 -m pip install torcheval && \
python3.12 -m pip cache purge
+
+# Remove unused installation packages to free up disk space
+RUN apt-get remove --purge -y \
+ '^aspnet.*' '^dotnet-.*' '^llvm-.*' 'php.*' '^mongodb-.*' \
+ snapd google-chrome-stable microsoft-edge-stable firefox \
+ azure-cli google-cloud-sdk mono-devel powershell libgl1-mesa-dri || true
+RUN apt-get autoremove --purge -y
+RUN apt-get clean
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
