Github user mogthesprog commented on a diff in the pull request:
https://github.com/apache/spark/pull/16061#discussion_r102437781
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -596,6 +599,26 @@ object SparkSubmit extends CommandLineUtils {
}
}
+ if (isKubernetesCluster) {
--- End diff --
Hey Guys, really excited about this work by the way. :)
Just wondering if client mode from within a kubernetes cluster will be
supported? Not looking to add work, just curious.
I'm wondering whether we could check if the client IP is within the
kubernetes container CIDR (this info should be available via the kubernetes
API), instead of just blocking client mode altogether. This would support
Zeppelin/Jupyter instances running within kube connecting to spark. Which is a
big usecase in our case.
We could take the
[nodelist](https://kubernetes.io/docs/api-reference/v1/definitions/#_v1_nodelist)
and check if the client IP is in any of the
[v1.NodeSpec.PodCIDR](https://kubernetes.io/docs/api-reference/v1/definitions/#_v1_nodespec)
of any node. Although as k8s clusters get large (nodes>1000), this may become
a costly operation.
This would allow for client mode support, provided the client is on one of
the kube nodes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
