uranusjr opened a new pull request, #29008:
URL: https://github.com/apache/airflow/pull/29008
This takes a callable that would be invoked when the XComArg is resolved.
Once implemented an example DAG would look something like this:
```python
import json
value = ... # Get some JSON string maybe from API.
@task
def task2(obj: dict):
print(obj["id"])
task2(value.convert(json.loads))
```
Of course, this is already possible right now by putting `json.loads` in a
task, either merged into the task that received `value`, the one that does
processing (`task2`), or a separate one in between. But this is useful if both
the tasks before and after are classic operators (i.e. relatively cumbersome to
extend), and you don’t want an extra task for whatever reason.
Note that, unlike `map()`, since the callable is applied to the arg as a
whole, the return value may no longer be expandable. Therefore we would raise
an error if the value is used for task-mapping.
TODO:
- [ ] Correctly handle `NotExpandable` so the scheduler does not crash when
`convert` is used incorrectly.
- [ ] Tests.
- [ ] Documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]