Björn Wenzel created SPARK-24905:
------------------------------------
Summary: Spark 2.3 Internal URL env variable
Key: SPARK-24905
URL: https://issues.apache.org/jira/browse/SPARK-24905
Project: Spark
Issue Type: Bug
Components: Kubernetes
Affects Versions: 2.3.1
Reporter: Björn Wenzel
Currently the Kubernetes Master internal URL is hardcoded in the
Constants.scala file
([https://github.com/apache/spark/blob/85fe1297e35bcff9cf86bd53fee615e140ee5bfb/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Constants.scala#L75)]
In some cases these URL should be changed e.g. if the Certificate is valid for
another Hostname.
Is it possible to make this URL a property like:
spark.kubernetes.authenticate.driver.hostname?
Kubernetes The Hard Way maintained by Kelsey Hightower for example uses
kubernetes.default as hostname, this will produce again a
SSLPeerUnverifiedException.
Here is the use of the Hardcoded Host:
[https://github.com/apache/spark/blob/85fe1297e35bcff9cf86bd53fee615e140ee5bfb/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala#L52]
maybe this could be changed like the KUBERNETES_NAMESPACE property in Line 53.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
