shepherd44 commented on PR #72821: URL: https://github.com/apache/airflow/pull/72821#issuecomment-6098363933
Closing this draft — I'm not going to carry it through review. The underlying issue is real and I'll leave the analysis here in case anyone wants to pick it up: `invoke_defer_method` takes a shortcut when the pod already reached a terminal state before the operator got to defer (log line: `Skipping deferral as pod is already in a terminal state`). It calls `trigger_reentry(...)` inline instead of `self.defer(...)`, and discards the return value. `trigger_reentry()` is what produces the task result that Airflow pushes to XCom as `return_value` when `do_xcom_push=True`, so the task succeeds but `return_value` never reaches XCom — only `pod_name` and `pod_namespace` are present, because `execute_async` pushes those separately. Three `return`s are missing: `execute` → `execute_async` → `invoke_defer_method`. The sync path has them, which is why only the deferrable path is affected. Short-lived pods hit this often. We worked around it downstream by setting `deferrable=False` on tasks that use `do_xcom_push`. Note `EksPodOperator` (amazon) overrides `invoke_defer_method` with the same shortcut, so a fix here would not cover it — that was #73906, also closed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
