Nataneljpwd commented on PR #60717: URL: https://github.com/apache/airflow/pull/60717#issuecomment-3801552716
> > How can a spark application create another driver pod? I do not see how it happens without the driver pod failing, in which case we can just select the pod by status. > > You are correct here. The pod may be recreated only if the driver pod fails. This still does not prevent 2 pods with the same label existing at the same time. If the driver pod fails, why can't we check the status in the query instead of doing a fallback? we can have a harder more deterministic behavior here where we check if the pod is not in a terminal state when reataching allong the label selection. Maybe I am missing something but this seems to be the better and less error-prone solution -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
