avantgardnerio opened a new pull request, #2251:
URL: https://github.com/apache/datafusion-ballista/pull/2251
## Summary
Additive Docker image changes for S3-backed EKS benchmark runs, plus wiring
the
`h2o` binary (added in #2230) into the benchmarks image.
- **`ballista-scheduler` + `ballista-executor`**: install `ca-certificates`.
Without them, DataFusion's `object_store` S3 client fails TLS verification
with
`InvalidCertificate(UnknownIssuer)`, and STS `AssumeRoleWithWebIdentity`
(IRSA-authenticated S3 reads on EKS) fails the same way. Ubuntu's minimal
image doesn't populate `/etc/ssl/certs`.
- **`ballista-benchmarks`**: same `ca-certificates` rationale, plus
`COPY target/release/h2o /root/h2o` so the image bundles both the tpch and
h2o binaries. The h2o binary shipped in #2230 but was never wired into the
Dockerfile.
- **`.dockerignore`**: whitelist `target/release{,-nonlto}/h2o` so the
benchmarks build context sees the h2o binary alongside tpch/etc. Missing
whitelist entry means the `COPY` line above errors out during build.
Andy's TPC-H bench workflow is unaffected — every change is additive.
## Test plan
- [ ] `docker build -f dev/docker/ballista-scheduler.Dockerfile .` succeeds
- [ ] `docker build -f dev/docker/ballista-executor.Dockerfile .` succeeds
- [ ] `docker build -f dev/docker/ballista-benchmarks.Dockerfile .` succeeds
- [ ] `docker run --rm <benchmarks-image> ls /root/h2o` prints the binary
path
- [ ] `docker run --rm <executor-image> ls /etc/ssl/certs | wc -l` > 0
- [ ] EKS run: h2o Q1 reads from an `s3://...` path without TLS errors
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
