Arunodoy18 opened a new pull request, #61024: URL: https://github.com/apache/airflow/pull/61024
In Apache Airflow 3.x, logs emitted from DAG.on_failure_callback (e.g. logging.* or print()) are silently dropped because the callback executes outside of a TaskInstance context and no log handler is attached. This PR ensures that logs produced inside DAG.on_failure_callback are written to scheduler logs, making DAG-level failure callbacks observable and debuggable without changing existing callback semantics. Problem: DAG.on_failure_callback runs in scheduler/DAG-processing code paths No task or scheduler logging context is active at execution time User-emitted logs inside the callback are lost Side effects (file I/O, external calls) still occur, making the issue non-obvious Task-level on_failure_callback does not have this issue, which highlights an inconsistency specific to DAG-level callbacks. Solution: The callback invocation is wrapped in a scheduler-scoped logging context so that: Logs emitted inside DAG.on_failure_callback are written to scheduler logs No task logs are created Callback execution order and behavior remain unchanged No public ApIs or configuration options are added Tests: Added a test that defines a DAG with on_failure_callback Forces a DAG failure Emits a log line inside the callback Asserts that the log is captured by the scheduler logger Backward Compatibility No breaking changes No impact on task-level callbacks Existing DAGs continue to behave the same, with improved observability Related Issue Closes #60969 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
