Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21748
> I don't think you can back a service with a selector that's a pod's name,
but someone with more knowledge of the Service API might be able to correct me
here. I was under the impression one had to use labels.
Yes, the service gets its endpoints by matching its label selector against
labels on the pods so it's critical to have matching labels. Another tenable
solution is for the driver backend code to get the pod object of the driver and
sets the label selector of the service based on the labels on the driver pod.
> don't think we should be special-casing Kubernetes here as being any
different from the other cluster managers. The main point of client mode is
that the driver is running locally and we make no assumptions about network
connectivity. Deploying notebooks on other cluster managers which have network
isolation between the driver and the executors would require the user to
manually configure their own connectivity too.
I would back-step a bit and argue that Kubernetes is unique in this regard
because of the need of a service for in-cluster client mode to work and the
service is a Kubernetes resource. For this reason, I don't think it makes no
sense for us to do a bit more for the users if technically it's not impossible
to do so.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
