[
https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919346#comment-16919346
]
Björn commented on SPARK-28444:
-------------------------------
We're running into the same issue. As I'm developing with a local
ansible/Vagrant setup (Kubernetes deployed through kubeadm, 3 nodes) I did some
testings on different versions with SparkPi example (Spark 2.4.3). My results
were:
* spark-submit in cluster and client mode works fine in local docker-desktop
running Kubernetes 1.14.3
* spark-submit in client mode works fine for Kubernetes 1.15.3 in the Vagrant
multinode cluster
* spark-submit in cluster mode does not work for Kubernetes 1.15.3,
1.14.3,1.13.10 in the Vagrant multinode cluster
spark-submit in cluster mode starts the driver, spawns the executors but fails
when trying to watch the pod with HTTP 403 Exception with an empty message
(esp. not complaining about permissions). The log is more or less the same as
posted above.
I think neither the compatibility nor the permissions (as executor pods can be
created with the service account, spark sa has cluster role) are the cause for
this.
Does anyone have ideas how to further debug this?
> Bump Kubernetes Client Version to 4.3.0
> ---------------------------------------
>
> Key: SPARK-28444
> URL: https://issues.apache.org/jira/browse/SPARK-28444
> Project: Spark
> Issue Type: Dependency upgrade
> Components: Kubernetes
> Affects Versions: 3.0.0, 2.4.3
> Reporter: Patrick Winter
> Priority: Major
>
> Spark is currently using the Kubernetes client version 4.1.2. This client
> does not support the current Kubernetes version 1.14, as can be seen on the
> [compatibility
> matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]].
> Therefore the Kubernetes client should be bumped up to version 4.3.0.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
