github-actions[bot] opened a new pull request, #71608: URL: https://github.com/apache/airflow/pull/71608
* Fix one bad callback request crashing the Dag processor and dropping the rest A callback request that hits the documented race condition (the Dag or task was removed between scheduling the callback and parsing the file) raises out of _execute_callbacks, killing the parse subprocess. The remaining requests in the batch were already popped from the manager's queue, so on_failure_callback, on_retry_callback, and failure-email requests behind the bad one are lost permanently. Isolate each request so a failure is logged and the rest still execute, and route the last_ti task lookup through _get_dag_with_task so a removed task surfaces as the race-condition error instead of a raw TaskNotFound. * Serialize callback request once per loop iteration Avoids a second to_json() in the exception handler, where a serialization failure would mask the original error and abort the batch the handler exists to protect. * Address review: keep run conf out of logs and deliver Dag callbacks when only the task is gone The last_ti task lookup only enriches the callback context, so a task removed since the run should downgrade to the minimal context (as produce_dag_callback already does for an unrepresentable last_ti) instead of costing the user the callback. The failure log leaked user-supplied dag_run.conf at ERROR via the serialized request and carried no structured identifiers; it now excludes context_from_server and binds dag_id/run_id/ti_id so log filters can match the line. * fixup! Address review: keep run conf out of logs and deliver Dag callbacks when only the task is gone (cherry picked from commit 13b2969c783e2e30ee2290ac9902fc8858b8f12e) Co-authored-by: Ephraim Anierobi <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
