fediabdelhedi opened a new pull request, #56710: URL: https://github.com/apache/airflow/pull/56710
Fix: avoid KeyError in _execute_task_callbacks when DAG missing from DagBag (#56701) ## Problem Fixes #56701 The `_execute_task_callbacks` function directly indexed `dagbag.dags[request.ti.dag_id]` and raised `KeyError` when the DAG was missing from the DagBag (race condition: DAG removed or not yet parsed). This caused the dag-processor to crash with: KeyError: '<dag_id>' File "airflow/dag_processing/processor.py", line 289, in _execute_task_callbacks dag = dagbag.dags[request.ti.dag_id] ## Solution - Perform defensive lookup using `dagbag.get_dag(dag_id)` or `dagbag.dags.get(dag_id)` (returns `None` if missing) - Handle missing `request.ti` or `request.ti.dag_id` gracefully - Log clear warning with context (dag_id, request) when DAG is not found - Skip callbacks and return early instead of crashing the processor ## Impact - Prevents processor crashes on transient conditions (DAG removal, parse delays) - Improves robustness and stability of dag-processor - Maintains existing behavior when DAG is present Closes: #56701 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
