jason810496 commented on code in PR #76826:
URL: https://github.com/apache/airflow/pull/76826#discussion_r4595843837
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -553,6 +553,36 @@ def head_xcom(
response.headers["Content-Range"] = f"map_indexes {count}"
+def _resolve_requested_map_index(
+ *,
+ dag_id: str,
+ run_id: str,
+ task_id: str,
+ params: GetXcomFilterParams,
+ session: Session,
+ dag_bag: DBDagBag,
+) -> int | None:
+ """
+ Return the map index to read, or None for every instance of a mapped task.
+
+ An omitted map index is resolved by the producing task's definition in the
+ Dag version its run uses: a mapped operator, or a task in a mapped task
+ group, is mapped however many instances it expanded to. A task that cannot
+ be found reads -1, so the read reports the XCom as not found.
+ """
+ if params.map_index is not None:
+ return params.map_index
+ if params.offset is not None:
+ return -1 # None reads a mapped task's list.
+ try:
+ task = TaskCoordinateResolver(dag_bag, session).get_task(dag_id,
run_id, task_id)
+ except AmbiguousProducerError as error:
+ raise HTTPException(status.HTTP_409_CONFLICT, str(error)) from error
+ except (TaskNotFound, ValueError):
+ return -1
+ return None if task.get_needs_expansion() else -1
Review Comment:
Thanks for the explanation, then I think it's fine to resolve the sequense.
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -608,15 +678,30 @@ def get_xcom(
),
},
)
+ if map_index is None:
+ mapped_read = _build_xcom_read(
+ dag_id=dag_id,
+ run_id=run_id,
+ task_id=task_id,
+ key=key,
+ session=session,
+ resolver=resolver,
+ token=token,
+ include_prior_dates=params.include_prior_dates,
+ previous_iteration=params.previous_iteration,
+ sequence=True,
+ )
+ values = _read_xcom_slice(mapped_read, start=None, stop=None,
step=None, session=session)
+ return XComResponse(key=key, value=values)
Review Comment:
With a custom XCom backend, an omitted map index returns storage references
instead of values.
`XCom.get_one` loads `value` once as a single XCom, while `get_all` loads
each item.
```python
# [core] xcom_backend =
airflow.providers.common.io.xcom.backend.XComObjectStorageBackend
# [common.io] xcom_objectstorage_threshold = 0
@task
def produce(i):
return {"result": i}
@task
def consume(ti):
one = ti.xcom_pull(task_ids="produce", map_indexes=1)
every = ti.xcom_pull(task_ids="produce", map_indexes=None)
produce.expand(i=[0, 1]) >> consume()
```
| | `one` | `every` |
|---|---|---|
| Before (API 2026-10-30) | `{"result": 1}` | `None` |
| After (head) | `{"result": 1}` | `["file:///.../6dd47fbe-...",
"file:///.../a066364b-..."]` |
| Expected | `{"result": 1}` | `[{"result": 0}, {"result": 1}]` |
An explicit map index still loads the stored value, so only the omitted case
breaks.
An unmapped task returning `[1, 2]` and this mapped read both arrive as
`{"value": [...]}`.
So `get_one` can't tell whether to load the list once or load each item.
Could the response mark a mapped read, so the SDK loads each item like
`get_all`?
The client-side fallback in #76825 has the same gap:
https://github.com/apache/airflow/pull/76825#discussion_r4595966210
##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -553,6 +553,36 @@ def head_xcom(
response.headers["Content-Range"] = f"map_indexes {count}"
+def _resolve_requested_map_index(
+ *,
+ dag_id: str,
+ run_id: str,
+ task_id: str,
+ params: GetXcomFilterParams,
+ session: Session,
+ dag_bag: DBDagBag,
+) -> int | None:
+ """
+ Return the map index to read, or None for every instance of a mapped task.
+
+ An omitted map index is resolved by the producing task's definition in the
+ Dag version its run uses: a mapped operator, or a task in a mapped task
+ group, is mapped however many instances it expanded to. A task that cannot
+ be found reads -1, so the read reports the XCom as not found.
+ """
+ if params.map_index is not None:
+ return params.map_index
+ if params.offset is not None:
+ return -1 # None reads a mapped task's list.
+ try:
+ task = TaskCoordinateResolver(dag_bag, session).get_task(dag_id,
run_id, task_id)
+ except AmbiguousProducerError as error:
Review Comment:
This is even better, thanks for the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]