will-byrne-cardano commented on issue #26760: URL: https://github.com/apache/airflow/issues/26760#issuecomment-1673495862
> Hi All! Hi @josh-fell, If the same callback is applied to all tasks, I think, it would be triggered by each task failure instead of being executed only once at the DAG failure. Please share your thoughts. > > > If you want to gain the context from the failed task, you can set [`on_failure_callback` at the task level](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/models/baseoperator/index.html?#airflow.models.baseoperator.BaseOperator) or, I presume you'd like the same callback to apply to all tasks, within [`default_args`](https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html?highlight=default_args#default-arguments). > > It would be nice to be capable of getting the earliest failed task instance from the context or even all the task instances that failed. > > Please advise. Hi @cklimkowski One possible workaround you could use that I have employed in some of my dags to find "the earliest failed task instance from the context or even all the task instances that failed" is to use the following pattern when designing callback functions: ```python def failure_callback(context): dag_run = context.get("dag_run") tis = dag_run.get_task_instances() failed_instances = [ti for ti in tis if ti.state == "failed"] failed_instance_ids [ti.task_id for ti in failed_instances] ... # add further logic e.g. take first element for first failed task ``` This should always get at least the first failed task instance. However, if a failed task triggers a dag failure and there are still other tasks running, and those tasks end up failing, it won't pick these up as their state will be `running` when the callback fires. A potential workaround (that I've not tested) if you need all failed is to implement a retry with a backoff on your callback that waits until no tasks have the state `running`. This could lead to complications with undead tasks though, so be sure to set some limit on number of retries if you are to do this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
