This is an automated email from the ASF dual-hosted git repository.
avantgardnerio pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-ballista.git
The following commit(s) were added to refs/heads/main by this push:
new 1b10233a5 chore(docker): ca-certificates + h2o binary in benchmarks
image (#2251)
1b10233a5 is described below
commit 1b10233a5111ef44919af15253199cf6c49b8a9f
Author: Brent Gardner <[email protected]>
AuthorDate: Fri Aug 7 13:47:25 2026 -0600
chore(docker): ca-certificates + h2o binary in benchmarks image (#2251)
* include h2o
* chore(docker): ca-certificates + h2o binary in benchmarks image
Additive changes needed for S3-backed benchmark runs and image parity
with the h2o binary that shipped in #2230.
- ballista-scheduler + ballista-executor: install `ca-certificates`.
Without them, TLS handshakes for S3 (via object_store) and STS
(AssumeRoleWithWebIdentity for IRSA-authenticated S3 reads) fail
with `InvalidCertificate(UnknownIssuer)`.
- ballista-benchmarks: same ca-certificates rationale, plus
`COPY target/release/h2o /root/h2o` so the image bundles both tpch
and h2o binaries. The h2o binary was added in #2230 but never wired
into the Dockerfile.
- .dockerignore: whitelist `target/release{,-nonlto}/h2o` so the
benchmarks build context sees the h2o binary alongside tpch/etc.
Whitelist is where the other binaries already live; without it,
docker build errors on the new COPY line.
Andy's TPC-H bench workflow is unaffected — every change is additive.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
---
.dockerignore | 2 ++
dev/docker/ballista-benchmarks.Dockerfile | 9 +++++++++
dev/docker/ballista-executor.Dockerfile | 7 +++++++
dev/docker/ballista-scheduler.Dockerfile | 8 ++++++++
4 files changed, 26 insertions(+)
diff --git a/.dockerignore b/.dockerignore
index f7fbbe56b..dfc8b5356 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -13,7 +13,9 @@ CHANGELOG.md
!target/release/ballista-executor
!target/release/ballista-cli
!target/release/tpch
+!target/release/h2o
!target/release-nonlto/ballista-scheduler
!target/release-nonlto/ballista-executor
!target/release-nonlto/ballista-cli
!target/release-nonlto/tpch
+!target/release-nonlto/h2o
diff --git a/dev/docker/ballista-benchmarks.Dockerfile
b/dev/docker/ballista-benchmarks.Dockerfile
index da7da429e..dd55ad8a8 100644
--- a/dev/docker/ballista-benchmarks.Dockerfile
+++ b/dev/docker/ballista-benchmarks.Dockerfile
@@ -21,6 +21,14 @@ LABEL
org.opencontainers.image.source="https://github.com/apache/datafusion-ball
LABEL org.opencontainers.image.description="Apache DataFusion Ballista
Distributed SQL Query Engine"
LABEL org.opencontainers.image.licenses="Apache-2.0"
+# ca-certificates so DataFusion's object_store S3 client can verify TLS
+# certs. Ubuntu's minimal image doesn't populate /etc/ssl/certs, and
+# reads from s3:// paths fail with InvalidCertificate(UnknownIssuer)
+# otherwise.
+RUN apt-get update && \
+ apt-get install -y --no-install-recommends ca-certificates && \
+ rm -rf /var/lib/apt/lists/*
+
ARG RELEASE_FLAG=release
ENV RELEASE_FLAG=${RELEASE_FLAG}
@@ -30,6 +38,7 @@ ENV RUST_BACKTRACE=full
COPY target/${RELEASE_FLAG}/ballista-scheduler /root/ballista-scheduler
COPY target/${RELEASE_FLAG}/ballista-executor /root/ballista-executor
COPY target/${RELEASE_FLAG}/tpch /root/tpch
+COPY target/${RELEASE_FLAG}/h2o /root/h2o
COPY benchmarks/run.sh /root/run.sh
COPY benchmarks/queries/ /root/benchmarks/queries
diff --git a/dev/docker/ballista-executor.Dockerfile
b/dev/docker/ballista-executor.Dockerfile
index 091a887c8..d64333131 100644
--- a/dev/docker/ballista-executor.Dockerfile
+++ b/dev/docker/ballista-executor.Dockerfile
@@ -23,6 +23,13 @@ LABEL org.opencontainers.image.licenses="Apache-2.0"
ARG RELEASE_FLAG=release
+# ca-certificates so DataFusion's object_store S3 client can verify TLS
+# certs at query time (executors do the parquet reads). Reads from
+# s3:// fail with InvalidCertificate(UnknownIssuer) otherwise.
+RUN apt-get update && \
+ apt-get install -y --no-install-recommends ca-certificates && \
+ rm -rf /var/lib/apt/lists/*
+
ENV RELEASE_FLAG=${RELEASE_FLAG}
ENV RUST_LOG=info
ENV RUST_BACKTRACE=full
diff --git a/dev/docker/ballista-scheduler.Dockerfile
b/dev/docker/ballista-scheduler.Dockerfile
index 169a76637..b3eadebbe 100644
--- a/dev/docker/ballista-scheduler.Dockerfile
+++ b/dev/docker/ballista-scheduler.Dockerfile
@@ -23,6 +23,14 @@ LABEL org.opencontainers.image.licenses="Apache-2.0"
ARG RELEASE_FLAG=release
+# ca-certificates so the scheduler can complete TLS handshakes at
+# plan-time. Applies to any outbound HTTPS: e.g. STS
+# AssumeRoleWithWebIdentity for IRSA-authenticated S3 reads during
+# schema inference, or fetching from HTTPS-hosted config sources.
+RUN apt-get update && \
+ apt-get install -y --no-install-recommends ca-certificates && \
+ rm -rf /var/lib/apt/lists/*
+
ENV RELEASE_FLAG=${RELEASE_FLAG}
ENV RUST_LOG=info
ENV RUST_BACKTRACE=full
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
