jscheffl commented on issue #37810:
URL: https://github.com/apache/airflow/issues/37810#issuecomment-2016560016
> This also opens the door for sending multiple things from one single
function if we allow `yield Output(...)`. I can think of future extensions that
the return value does not go to the XCom storage, but whatever is specified in
`outlets` directly (without needing to explicitly write data in the function).
There are a lot of opportunities.
Like to have this option. Also thought of this. Had the idea of "future
extension" as well with the primary intend to keep it simple first :-D
> That said, I think implementing the context-based approach is still a good
first step toward all this. Even with the more magical and convenient
return-as-metadata syntax, using a context variable is still explicit and may
be preferred by some. It is also easier to implement, and should be a good way
to start things rolling without getting into a ton of syntax design but focus
on the core feature here. So I’m going to start with that first.
Looking forward to a PR.
What do you think if maybe your any my proposal are both possible, mostly
based on how the used needs it? In many cases the extra information might not
be used at-all or with the existing mechanisms is totally fine. Something like:
```
@task(outlets=[Dataset("s3://my/data.json", extra_from_return=True)])
```
or
```
@task(outlets=[Dataset("s3://my/data.json", extra_from_xcom="dataset_key")])
```
..whereas in your notation with the `yield` you explicitly push the dataset,
I assume before task completion already. Instead of `uri` might be better to
define the target Dataset directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
