eladkal commented on PR #74458: URL: https://github.com/apache/airflow/pull/74458#issuecomment-6061708825
I am not convinced this is a bug. The `on_failure_callback` is designed for dag authors to run a function when failure occurs for business reasons. I don't think we ever designed it to accommodate for infrastructure failure like worker crash. This is why we say that tasks in Airflow should be idempotent. I suspect that someone took the `on_failure_callback` to do more that what it was designed for. We can always re-evaluate and maybe support the requested feature but that is something we need to think about thoroughly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
