Agree with Ash, And I think if we add it, it will detract people from doing the same thing in a better way.
I think what I really like about TaskFlow is that it actually does not need to use Jinja2 at all. Jinja2 as I see it, is a super-convoluted way of running (subset of) Python via strings. Not only dangerous from a security point of view but also has a number of deficiencies (no type validation really possible, no good support in IDES, limited set of filters to use and plenty of other limitations and problems). You could not avoid using Jinja2 in "classic" operators which are treated as black-box. And this is the only reason IMHO we use JINJA2 at all in Airflow. If you look at what features TaskFlow provides - one of its features is to make our code pure-Python, and Jinja2-less. Simply, if you want to dynamically derive a value - do it in Python code, not in JINJA2. If we think that we need a feature, rather than implementing it in Jinja. For me this is quite simple equation: * Classic Operators = Use Jinja2 * Task Flow = Do not use Jinja2 The example of Ash is a pure manifestation of that. J. On Mon, Aug 29, 2022 at 9:01 AM Ash Berlin-Taylor <[email protected]> wrote: > In this example I'd say it shouldn't be templated, as you can do it via > > @task > def get_templated_echo(ds): > return f'echo "The date is {ds}"' > > (This is nothing particular to XcomArg either, just the way Xcom had > always worked.) > > I don't think we can change the current/default behaviour without it > likely being a breaking change to someone's workflow. > > If there's enough desire and a good enough use case we can add a way to > make it something you can turn on via one way or another. > > -ash > > > > On 29 August 2022 07:46:01 BST, Tzu-ping Chung <[email protected]> > wrote: >> >> Context: https://github.com/apache/airflow/issues/26016 >> >> Currently if a value is passed to a downstream via XComArg, e.g. >> >> @taskdef get_templated_echo(): return 'echo "The date is {{ ds }}"' >> BashOperator(task_id="echo_task", bash_command=get_templated_echo()) >> >> >> Then this value does not go through Jinja2 templating, i.e. the >> BashOperator above would print The date is {{ ds }} instead of The date >> is 2022-08-29. The question here is, is this a conscious design >> decision, and is there reasoning behind it? Personally I can see arguments >> from both sides, but some context would be very appreciated. >> >> And, if there wasn’t an explicit decision, should template-rendering an >> XComArg be supported? >> >> TP >> >
