jkleckner commented on pull request #29496: URL: https://github.com/apache/spark/pull/29496#issuecomment-685104201
@shockdm This is an attempt to backport the master branch fix which has that behavior [1]. Also this works fine in our environment for a while now. As mentioned above [2], we are using further than v2.4.6 [3] since we wanted to bring in as many fixes as possible. We launch using the spark-operator-on-k8s [4] mechanism with this version [5] and have not tried any other launch method. So it is quite possible that there are issues with directly launching - how do you launch? [1] https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/LoggingPodStatusWatcher.scala#L114 [2] https://github.com/apache/spark/pull/29496#issuecomment-682251482 [3] https://gitlab.com/jkleckner/spark/-/tags/v2.4.7-cphy1 [4] https://github.com/GoogleCloudPlatform/spark-on-k8s-operator [5] https://gitlab.com/jkleckner/spark-on-k8s-operator/-/commit/f30dd1535220813315eea282a3d868aeb14cacb3 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
