This is an automated email from the ASF dual-hosted git repository.
yikun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-docker.git
The following commit(s) were added to refs/heads/master by this push:
new 95f5a1f [SPARK-40833] Cleanup apt lists cache
95f5a1f is described below
commit 95f5a1f3e846ad3b6550e151fa76b70f6fe0b946
Author: Yikun Jiang <yikunk...@gmail.com>
AuthorDate: Wed Oct 19 10:17:58 2022 +0800
[SPARK-40833] Cleanup apt lists cache
### What changes were proposed in this pull request?
Remove unused apt lists cache and apply `./add-dockerfiles.sh 3.3.0`
### Why are the changes needed?
Clean cache to reduce docker image size.
This is also
[recommanded](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run)
by docker community:
```
$ docker run --user 0:0 -ti apache/spark bash
root5d1ca347279e:/opt/spark/work-dir# ls /var/lib/apt/lists/
auxfiles
lock
deb.debian.org_debian_dists_bullseye-updates_InRelease
partial
deb.debian.org_debian_dists_bullseye-updates_main_binary-arm64_Packages.lz4
security.debian.org_debian-security_dists_bullseye-security_InRelease
deb.debian.org_debian_dists_bullseye_InRelease
security.debian.org_debian-security_dists_bullseye-security_main_binary-arm64_Packages.lz4
deb.debian.org_debian_dists_bullseye_main_binary-arm64_Packages.lz4
root5d1ca347279e:/opt/spark/work-dir# du --max-depth=1 -h
/var/lib/apt/lists/
4.0K /var/lib/apt/lists/partial
4.0K /var/lib/apt/lists/auxfiles
17M /var/lib/apt/lists/
```
### Does this PR introduce _any_ user-facing change?
Yes in some level, image size is reduced.
### How was this patch tested?
K8s CI passed
Closes #14 from Yikun/clean-apt-list.
Authored-by: Yikun Jiang <yikunk...@gmail.com>
Signed-off-by: Yikun Jiang <yikunk...@gmail.com>
---
3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile | 3 ++-
3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile | 3 ++-
3.3.0/scala2.12-java11-r-ubuntu/Dockerfile | 3 ++-
3.3.0/scala2.12-java11-ubuntu/Dockerfile | 3 ++-
Dockerfile.template | 3 ++-
5 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile
b/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile
index 5dbc973..be9cbb0 100644
--- a/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile
+++ b/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile
@@ -38,7 +38,8 @@ RUN set -ex && \
ln -sv /bin/bash /bin/sh && \
echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \
chgrp root /etc/passwd && chmod ug+rw /etc/passwd && \
- rm -rf /var/cache/apt/*
+ rm -rf /var/cache/apt/* && \
+ rm -rf /var/lib/apt/lists/*
# Install Apache Spark
# https://downloads.apache.org/spark/KEYS
diff --git a/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile
b/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile
index 85e06ce..096c7eb 100644
--- a/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile
+++ b/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile
@@ -37,7 +37,8 @@ RUN set -ex && \
ln -sv /bin/bash /bin/sh && \
echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \
chgrp root /etc/passwd && chmod ug+rw /etc/passwd && \
- rm -rf /var/cache/apt/*
+ rm -rf /var/cache/apt/* && \
+ rm -rf /var/lib/apt/lists/*
# Install Apache Spark
# https://downloads.apache.org/spark/KEYS
diff --git a/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile
b/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile
index 753d585..2e085a2 100644
--- a/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile
+++ b/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile
@@ -35,7 +35,8 @@ RUN set -ex && \
ln -sv /bin/bash /bin/sh && \
echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \
chgrp root /etc/passwd && chmod ug+rw /etc/passwd && \
- rm -rf /var/cache/apt/*
+ rm -rf /var/cache/apt/* && \
+ rm -rf /var/lib/apt/lists/*
# Install Apache Spark
# https://downloads.apache.org/spark/KEYS
diff --git a/3.3.0/scala2.12-java11-ubuntu/Dockerfile
b/3.3.0/scala2.12-java11-ubuntu/Dockerfile
index 1e4c604..5858e2d 100644
--- a/3.3.0/scala2.12-java11-ubuntu/Dockerfile
+++ b/3.3.0/scala2.12-java11-ubuntu/Dockerfile
@@ -34,7 +34,8 @@ RUN set -ex && \
ln -sv /bin/bash /bin/sh && \
echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \
chgrp root /etc/passwd && chmod ug+rw /etc/passwd && \
- rm -rf /var/cache/apt/*
+ rm -rf /var/cache/apt/* && \
+ rm -rf /var/lib/apt/lists/*
# Install Apache Spark
# https://downloads.apache.org/spark/KEYS
diff --git a/Dockerfile.template b/Dockerfile.template
index 2001281..a031b16 100644
--- a/Dockerfile.template
+++ b/Dockerfile.template
@@ -44,7 +44,8 @@ RUN set -ex && \
ln -sv /bin/bash /bin/sh && \
echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \
chgrp root /etc/passwd && chmod ug+rw /etc/passwd && \
- rm -rf /var/cache/apt/*
+ rm -rf /var/cache/apt/* && \
+ rm -rf /var/lib/apt/lists/*
# Install Apache Spark
# https://downloads.apache.org/spark/KEYS
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
