Jeffwan commented on issue #27347: [SPARK-30626][K8S] Add SPARK_APPLICATION_ID into driver pod env URL: https://github.com/apache/spark/pull/27347#issuecomment-578055197 Run integration-tests locally and notice the problem. Seems ``` + [[ docker.io/kubespark == gcr.io* ]] + /Users/shjiaxin/Github/spark/bin/docker-image-tool.sh -r docker.io/kubespark -t 82565652-8092-4BF5-A804-9FCDC6CCE5AA push docker.io/kubespark/spark:82565652-8092-4BF5-A804-9FCDC6CCE5AA image not found. Skipping push for this image. docker.io/kubespark/spark-py:82565652-8092-4BF5-A804-9FCDC6CCE5AA image not found. Skipping push for this image. docker.io/kubespark/spark-r:82565652-8092-4BF5-A804-9FCDC6CCE5AA image not found. Skipping push for this image. + cd - ``` Notice there's pod failure in my cluster. ``` kubectl get pods -n -n c99e1fb3b1b04171baf8f24b0f4a6666 NAME READY STATUS RESTARTS AGE spark-test-app-2559fdde4ab749dfb45b03a59bdfed68 0/1 ErrImagePull 0 10s ``` Here's the pod spec. It's pretty clear because pods image doesn't push to registry and pod can not fetch image correctly. But I am not sure if this has exact same reason to 2 failures in CI. Because of missing container image, all my tests failed. If there're only 2 failures, it's probably due to some other problems. ``` ---- ------ ---- ---- ------- Normal Scheduled 59s default-scheduler Successfully assigned c99e1fb3b1b04171baf8f24b0f4a6666/spark-test-app-a6037cf7dfa34ed9a500c1f752d82688 to ip-192-168-3-231.us-west-2.compute.internal Warning FailedMount 58s (x2 over 59s) kubelet, ip-192-168-3-231.us-west-2.compute.internal MountVolume.SetUp failed for volume "spark-conf-volume" : configmap "longlonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglong-1981f16fd6d3f76f-driver-conf-map" not found Normal Pulling 18s (x3 over 57s) kubelet, ip-192-168-3-231.us-west-2.compute.internal Pulling image "docker.io/kubespark/spark:82565652-8092-4BF5-A804-9FCDC6CCE5AA" Warning Failed 17s (x3 over 56s) kubelet, ip-192-168-3-231.us-west-2.compute.internal Failed to pull image "docker.io/kubespark/spark:82565652-8092-4BF5-A804-9FCDC6CCE5AA": rpc error: code = Unknown desc = Error response from daemon: manifest for kubespark/spark:82565652-8092-4BF5-A804-9FCDC6CCE5AA not found Warning Failed 17s (x3 over 56s) kubelet, ip-192-168-3-231.us-west-2.compute.internal Error: ErrImagePull Normal BackOff 5s (x3 over 55s) kubelet, ip-192-168-3-231.us-west-2.compute.internal Back-off pulling image "docker.io/kubespark/spark:82565652-8092-4BF5-A804-9FCDC6CCE5AA" Warning Failed 5s (x3 over 55s) kubelet, ip-192-168-3-231.us-west-2.compute.internal Error: ImagePullBackOff ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
