Kaicheng Zhou created SPARK-59212:
-------------------------------------
Summary: K8s integration tests fail because minikube 1.39.0
defaults to the containerd runtime
Key: SPARK-59212
URL: https://issues.apache.org/jira/browse/SPARK-59212
Project: Spark
Issue Type: Bug
Components: Project Infra
Affects Versions: 4.4.0
Reporter: Kaicheng Zhou
Fix For: 4.4.0
The Kubernetes integration test job fails during Docker image build, before any
Spark test runs:
{code:java}
#0 building with "default" instance using docker-container driver
#1 pulling image moby/buildkit:buildx-stable-1
#1 ERROR: failed to inspect pulled image moby/buildkit:buildx-stable-1: Error
response from daemon: 404 page not found
ERROR: failed to build: ...
Failed to build Spark JVM Docker image{code}
minikube 1.39.0 changed the default container runtime to containerd. The
workflow runs {{eval $(minikube docker-env)}} and then builds the Spark images,
but minikube reports {{Using the docker-env command with the containerd runtime
is a highly experimental feature}} — the Docker CLI is pointed at
something that is not a real dockerd. buildx can no longer use the in-daemon
{{docker}} driver, silently falls back to {{{}docker-container{}}}, and the
buildkit bootstrap fails on the image-inspect call.
Runs on minikube 1.38.1 select the {{docker}} driver and pass; runs on 1.39.0
select {{docker-container}} and fail. The 1.38.1 runs already logged the
warning {{Starting v1.39.0, minikube will default to "containerd" container
runtime}} (minikube issue #21973). The Kubernetes version is not a factor —
the failure reproduces on both 1.36.0 and 1.37.0.
The workflow pins the {{setup-minikube}} action but not the minikube binary it
installs, so this broke with no change to Spark. Every job that builds
images through {{minikube docker-env}} is affected.
Proposed fix: set {{container-runtime: docker}} on each {{setup-minikube}}
step, or pin {{minikube-version}} to a known-good release.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
