dongjoon-hyun commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1583426436
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala:
##########
@@ -73,7 +73,8 @@ private[spark] class KubernetesDriverBuilder {
new HadoopConfDriverFeatureStep(conf),
new KerberosConfDriverFeatureStep(conf),
new PodTemplateConfigMapStep(conf),
- new LocalDirsFeatureStep(conf)) ++ userFeatures
+ new LocalDirsFeatureStep(conf),
+ new DriverIngressFeatureStep(conf)) ++ userFeatures
Review Comment:
To @pan3793 , I'm very careful about this PR because we consider `Having K8s
Ingress` is more vulnerable than having `Spark UI` because it increases the
boundary of exposure. `Spark UI` itself is not open to the other pods if you
design to isolate Spark pod-to-pod communication carefully.
In addition, some K8s implementations (and their security backend) increases
`K8s Ingress` creation. I expected a few minutes delays for only K8s ingress
entity creations. It could mean a delay of Spark Job time too if we design to
wait for K8s ingress.
Since we verified that the deletion is handled by K8s ownership correctly.
Let's double-check the following.
- What is the job start time for Option 1? There is no delay?
- If there is a delay, is there a way to mitigate it?
- If there is no delay because it is handled separately, what happens
during the time gap after the job start already and `K8s Ingress` is still
under construction. Are we in the same situation where `Option 2`'s `Cons 1`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
