This is an automated email from the ASF dual-hosted git repository.
yuqi1129 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new caa5a0bd5f [#12730] fix(docker): bind-mount packages/ in kerberos-hive
so the archives stop shipping in the image (#12731)
caa5a0bd5f is described below
commit caa5a0bd5f9d8e8bb8c9a07777fb2799f484e6dd
Author: Sujeito Operator <[email protected]>
AuthorDate: Wed Sep 2 03:22:54 2026 +0200
[#12730] fix(docker): bind-mount packages/ in kerberos-hive so the archives
stop shipping in the image (#12731)
### What changes were proposed in this pull request?
In `dev/docker/kerberos-hive/Dockerfile`, bind-mount `packages/` into
the three `RUN`s that untar it, and drop both the `COPY packages
/tmp/packages` that materialises it into a layer and the `RUN rm -rf
/tmp/packages` that cannot free it again. One file, net 2 lines shorter.
```dockerfile
RUN --mount=type=bind,source=packages,target=/tmp/packages \
tar -xz -C ${HADOOP_HOME} --strip-components 1 -f
/tmp/packages/${HADOOP_PACKAGE_NAME}
```
The mount target is the same `/tmp/packages`, so every `-f
/tmp/packages/${...}` argument is byte-identical to what is there today
and the three extracted trees are unchanged. A `--mount=type=bind` is
visible only while that `RUN` executes and is never committed to a
layer, so there is nothing left to `rm` and nothing to ship.
Two things I checked rather than assumed:
- **It is BuildKit.** `dev/docker/build-docker.sh` builds this image
with `docker buildx build --builder gravitino-builder ... -f Dockerfile
.`, so `RUN --mount=type=bind` is the stable frontend and no `# syntax=`
directive is needed. The published image's own layer history confirms it
from the other side: its `COPY` steps are recorded as `# buildkit`.
- **`source=packages` resolves to the right directory.** The same script
does `cd ${script_dir}/${component_type}` and passes `.` as the build
context, so the context is `dev/docker/kerberos-hive` and `packages` is
exactly what the `COPY` reads today. `hive-dependency.sh` populates it
in the same place, unchanged.
### Why are the changes needed?
The `COPY` writes the three install archives into a committed layer; the
`rm -rf` runs in a later layer and a later layer cannot reclaim bytes an
earlier one committed -- it only writes a whiteout on top. Both layers
ship, so the archives are pulled by everyone who pulls the image, on top
of their own already-extracted contents.
Read from Docker Hub on 2026-08-31 with an anonymous pull token -- no
login and no local build -- for
`apache/gravitino-ci:kerberos-hive-0.1.6`, `linux/amd64`:
| | compressed bytes |
|---|---|
| layer 7, `COPY packages /tmp/packages` | **506,263,481** |
| whole image, 63 layers | 1,414,100,417 |
| share of the image | **35.8%** |
To reproduce: take a pull token for `apache/gravitino-ci`, fetch the OCI
index for the tag, pick the `linux/amd64` manifest, and zip the config
blob's non-empty `history` entries against `manifest.layers`. Entry 7 is
the `COPY packages /tmp/packages # buildkit` step. (`docker history`
shows the same step; it reports the uncompressed size rather than the
pulled size, so the figures above are the smaller, pulled ones.)
Fix: #12730
### Does this PR introduce _any_ user-facing change?
No. No API, no property key, no configuration. The image built from this
Dockerfile contains the same files at the same paths --
`/usr/local/hadoop`, `/usr/local/hive` and the JDBC driver in
`${HIVE_HOME}/lib` are all extracted by the same `tar` invocations from
the same archives. The only difference is that the archives themselves
are no longer inside it.
### How was this patch tested?
Honestly, and with one gap I would rather name than paper over.
- **The claim about the current image is measured, not argued** -- the
table above is read from the published registry manifest for an
immutable tag, so it is checkable by anyone without trusting me or my
environment.
- **The edit is mechanical and pinned by a test on my side**: the three
`-f /tmp/packages/${...}` arguments are asserted byte-identical after
the patch, the `tar` invocation count is asserted unchanged, and the
three unrelated `COPY`s beside the removed one (`kdc.conf`, `kadm5.acl`,
`krb5.conf`) are asserted still present.
- **I did not build the image.** There is no Docker daemon in my
environment, so CI does the build I could not, and the after-figure will
come from the next publish rather than from me. `AGENTS.md` asks for a
unit test with all new logic; there is no new logic here and no
unit-test surface for a Dockerfile, which is why this section is a
description rather than a test name.
If a maintainer would rather keep the `COPY` for a build-cache reason I
am not seeing, please say so and close this -- it is one file, one
concern, and nothing else in the build depends on it.
---
*Opened by an autonomous AI agent. I wrote and tested this change end to
end; a human principal stands behind the work and is accountable for it.
Said up front because you should be able to weigh it before reading the
diff, not discover it afterwards — and because some projects would
rather not take AI contributions at all, which is a legitimate position:
say so and I will close this and stop.*
Signed-off-by: Sujeito Operator <[email protected]>
---
dev/docker/kerberos-hive/Dockerfile | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/dev/docker/kerberos-hive/Dockerfile
b/dev/docker/kerberos-hive/Dockerfile
index a556f7dba9..6c963940f6 100644
--- a/dev/docker/kerberos-hive/Dockerfile
+++ b/dev/docker/kerberos-hive/Dockerfile
@@ -66,7 +66,6 @@ RUN apt-get update && apt-get upgrade -y && apt-get install
--fix-missing -yq \
RUN mkdir /root/.ssh
RUN cat /dev/zero | ssh-keygen -q -N "" > /dev/null && cat
/root/.ssh/id_rsa.pub > /root/.ssh/authorized_keys
-COPY packages /tmp/packages
COPY kdc.conf /etc/krb5kdc/kdc.conf
COPY kadm5.acl /etc/krb5kdc/kadm5.acl
COPY krb5.conf /etc/krb5.conf
@@ -118,7 +117,8 @@ RUN echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >>
/etc/environment
################################################################################
# install hadoop
RUN mkdir ${HADOOP_HOME}
-RUN tar -xz -C ${HADOOP_HOME} --strip-components 1 -f
/tmp/packages/${HADOOP_PACKAGE_NAME}
+RUN --mount=type=bind,source=packages,target=/tmp/packages \
+ tar -xz -C ${HADOOP_HOME} --strip-components 1 -f
/tmp/packages/${HADOOP_PACKAGE_NAME}
# replace configuration templates
RUN rm -f ${HADOOP_CONF_DIR}/core-site.xml
@@ -139,7 +139,8 @@ ADD check-status.sh /tmp/check-status.sh
################################################################################
# install hive
RUN mkdir ${HIVE_HOME}
-RUN tar -xz -C ${HIVE_HOME} --strip-components 1 -f
/tmp/packages/${HIVE_PACKAGE_NAME}
+RUN --mount=type=bind,source=packages,target=/tmp/packages \
+ tar -xz -C ${HIVE_HOME} --strip-components 1 -f
/tmp/packages/${HIVE_PACKAGE_NAME}
ADD hive-site.xml ${HIVE_HOME}/conf/hive-site.xml
RUN mkdir ${HIVE_HOME}/conf1 && cp ${HIVE_HOME}/conf/* ${HIVE_HOME}/conf1/
ADD hive-site1.xml ${HIVE_HOME}/conf1/hive-site.xml
@@ -157,7 +158,8 @@ RUN sed -i "s/.*bind-address.*/bind-address = 0.0.0.0/"
/etc/mysql/mysql.conf.d/
################################################################################
# add mysql jdbc driver
-RUN tar -xz -C ${HIVE_HOME}/lib --strip-components 1 -f
/tmp/packages/${JDBC_DIVER_PACKAGE_NAME}
+RUN --mount=type=bind,source=packages,target=/tmp/packages \
+ tar -xz -C ${HIVE_HOME}/lib --strip-components 1 -f
/tmp/packages/${JDBC_DIVER_PACKAGE_NAME}
################################################################################
# add users and groups
@@ -174,10 +176,6 @@ RUN usermod -a -G mapred gravitino
RUN mkdir /home/gravitino
RUN chown -R gravitino:hadoop /home/gravitino
-################################################################################
-# removed install packages
-RUN rm -rf /tmp/packages
-
################################################################################
# expose port
EXPOSE 3306 9000 9083 10000 10002 50070 50075 50010 88 19083
