Pranaykarvi opened a new pull request, #70199: URL: https://github.com/apache/airflow/pull/70199
closes #70173 ### What Cross-DAG `xcom_pull(..., include_prior_dates=True)` returned `None` on Airflow 3.2.x / 3.3.0 (regression from 3.1.7). The upstream XCom is written correctly; only the cross-DAG read fails. ### Root cause `XComModel.get_many` derived the `include_prior_dates` cutoff with a correlated scalar subquery keyed on `DagRun.run_id == run_id AND DagRun.dag_id == cls.dag_id`. On a cross-DAG pull the request carries `dag_id = <target/upstream>` but `run_id = <caller's own run>`, so no matching `DagRun` exists (`cls.dag_id` is the upstream DAG). The subquery resolves to NULL, the `<=` predicate is NULL, every row is filtered out, and the SDK falls back to `None`. Same-DAG pulls worked because `run_id` and `dag_id` agree. The `cls.dag_id` scoping was intentional — it disambiguates `run_id` collisions across DAGs — so the fix keeps the caller's *DAG* rather than dropping the scoping. ### Fix `include_prior_dates` semantically means "XComs on or before the pulling task instance's own run date" (matching 3.1/2.x). The caller is identified server-side by the TI token, so the Execution API resolves the cutoff from the caller's own DAG run and passes it to `get_many` via a new optional `prior_dates_before`. When it's set, the query bounds directly on it; when it's `None`, the original correlated-subquery path is kept as a fallback for direct model-level callers. No SDK, schema, or API-version change — `include_prior_dates` was already parsed on both filter params. ### Behavioral note Because the cutoff now comes from the caller's TI, a *same-DAG* `xcom_pull(run_id=<earlier run>, include_prior_dates=True)` is bounded by the caller's run date rather than that explicit run's date. This restores pre-3.2 semantics (the reference was always the pulling TI's date), but it's a subtle change in that edge combination and is called out here intentionally. ### Tests - `test_xcom_get_many_from_prior_dates_cross_dag` (model): cross-DAG pull returns the upstream value. - `test_xcom_get_include_prior_dates_cross_dag` (route): end-to-end reproduction of #70173 through `get_xcom`. - Existing `test_xcom_get_many_from_prior_dates` and `..._scopes_run_id_to_dag` unchanged and passing. Both new tests fail before the fix (model: `TypeError` on `prior_dates_before`; route: `404`/`None`, the exact reported symptom). <img width="1800" height="1130" alt="Screenshot 2026-07-21 at 11 16 26 PM" src="https://github.com/user-attachments/assets/abd5a702-f968-49b2-b261-74ebfeedde7a"; /> --- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
