kaxil commented on issue #74660: URL: https://github.com/apache/airflow/issues/74660#issuecomment-6165199870
Makes sense, thanks. I'll put up the PR for `RetryAction.RESCHEDULE`, and a follow-up upstreaming the typed KPO exceptions would be welcome. One scoping note so this doesn't get mixed up with [AIP-97](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-97+Task+Failure+Classification) / AIP-122. `RESCHEDULE` is for when the task's own code decides that the external system it drives isn't ready, here the cluster KPO launches pods into. The Airflow worker itself ran fine, and the Dag author's policy makes the call. Disruption of the Airflow worker (executor pod preempted, Spot capacity reclaimed) is what AIP-97 classifies as `infra` and what AIP-122 grants admin-capped retries for. A retry policy never sees those, since the worker is gone. So `RESCHEDULE` won't touch `failure_kind`, and the docs won't describe it as an infrastructure failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
