stefanondisponibile opened a new issue #18982:
URL: https://github.com/apache/airflow/issues/18982
### Description
Hello, I'd love to discuss the idea of a new feature allowing users to
`push` to `XCOM` directly (not from a task).
### Use case/motivation
The motivation of this is allowing downstream tasks to pull XCOM values from
the upstream whenever upstream tasks failed for whatever reasons.
One very common pattern is the following:
**Task-1 (upstream)**:
- Upload something to an external repository (db, s3, ...).
- Push to XCOM some reference to that data (its id, name, path, ...)
**Task-2 (downstream)**:
- Pull from XCOM the reference to that data
- Do something with it
Now let's say **Task-1** takes *very* long (hours, or even more), and let's
say that **Task-1** completed the upload but for whatever reason if fails
before pushing the XCOM.
**Task-2** won't be able to pull any values, but being able to pass new
arguments to other tasks through manually updated XCOM may be useful (i.e.
time-saving). In this particular example, the user could update the **Task-1**
XCOM with the proper id.
What do you think are the drawbacks of this approach?
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
