wolvery commented on PR #59413: URL: https://github.com/apache/airflow/pull/59413#issuecomment-3720002914
> > What would be the steps to deprecate delete_succeeded_pods after introducing delete_non_failed_pods? > > Could I add a recommendation/deprecation warning? > > Why deprecating it? It's just another option and someone might want to use it. I don't see any reason to deprecate it. I believe this option could negatively impact other users, as it did us. That’s why I opened this issue: I think this option is somewhat risky when there are resource or namespace constraints in a Kubernetes cluster. We experienced situations where pods kept running after the Airflow worker finished and skipped them, which eventually led to no available machines to schedule new pods in our Kubernetes cluster. This situation persisted for a long time due to the Airflow retries we had in place, for example. I understand that the option itself is clear of the intent, but for users who enable it, it may not be clear that the Airflow worker can leave pods unsupervised in a Pending or Running state, and appear succesfull pods after their execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
