Github user echarles commented on the issue:
https://github.com/apache/spark/pull/21748
@mccheah If I compare with yarn-client with all nodes on the same LAN, we
introduce complexity here as the user has to ensure not only configuration, but
also deployment of a particular resource. I am not sure that yarn devops would
be happy to be obliged to deploy let's say a yarn container or yarn app for
each notebook before being able to use it. We do introduce complexity with
k8s-client compared to yarn-client.
About selecting the pod with labels, another approach I have taken is
simply using the name of the driver pod, a bit like I have done with the
following deployment (so no need to ensure labels - the ports are the ports
assigned by spark that the code can retrieve).
```
apiVersion: v1
kind: Service
metadata:
name: spark-driver-service
spec:
clusterIP: None
ports:
- port: 7077
name: spark-driver-port
- port: 10000
name: spark-driver-blockmanager-port
selector:
run: spark-pod
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
