sunchao commented on PR #57350: URL: https://github.com/apache/spark/pull/57350#issuecomment-5181139320
> @sunchao Thanks for the detailed review. Several of these findings share the same root cause: on hostNetwork, the port the driver actually binds to may differ from the configured one, so we should really be patching the Service's targetPort based on the actual bound port for both the driver Service and the UI Service. And I submit https://issues.apache.org/jira/browse/SPARK-58499. Do you think we should continue this? @zhengchenyu Yes, I think we should continue. This PR is useful on its own because it provides a dedicated UI Service, while SPARK-58499 can address the broader driver-Service reconciliation problem. One thing to clarify is that `spark.ui.port=0` still produces an invalid driver Pod, so either fix that here or avoid claiming support until SPARK-58499 resolves it. For this PR, we should also make sure the new UI Service works in driver-only mode and is properly cleaned up when the UI stops. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
