nickmyatt commented on PR #41513:
URL: https://github.com/apache/airflow/pull/41513#issuecomment-2297990613
I see that there is indeed no specific mention of combining `@dag` and
`Param` in the docs, but I got the idea to try it by looking at documentation
examples of `@dag` and `DAG(params=...)`, and was then surprised to find that
the "obvious" combination of those examples doesn't work as I would have
expected :)
Currently I am using a workaround that looks like
```
@task
def unwrap_param(param):
if isinstance(param, Param):
return param.value
return param
@dag(schedule="* * * * * ",
start_date=datetime.datetime.fromisoformat("2024-01-01"))
def test_dag(
example_array_param=Param(["foo", "bar"], description="An example of an
array parameter"),
):
example_array_param = unwrap_param(example_array_param)
log_array_param(example_array_param)
```
which does the same thing as the patch, but requires an extra `unwrap_param`
task in every DAG to do the "unwrapping".
I tried your suggestion, and the task prints out
```
{
{
p
a
r
a
m
s
.
e
x
a
m
p
l
# and so on ....
```
Looking at the implementation of `@dag`, I would guess that this is because
`dag_obj.param`
https://github.com/apache/airflow/blob/main/airflow/models/dag.py#L3887
instantiates a `DagParam`, which clobbers any existing param with that name
https://github.com/apache/airflow/blob/563bc494ab5c610c46a60b2fe6beed742e3d588e/airflow/models/param.py#L325.
Thus
```
params={
"example_array_param": Param(["foo", "bar"], description="An example
of an array parameter"),
},
```
has no effect. Moreover I'm not sure how Jinja templates actually interact
with TaskFlow-style parameter passing? I suspect they just don't get resolved.
In general I'm keen to avoid string templating (which is why I am using
`@dag`), because
- it doesn't work with static analysis
- it requires `render_template_as_native_obj` to be useful, which looks like
it needs to be turned on for the entire DAG, and so potentially changes the
behaviour of any existing templates.
---
All that said, what would be needed to consider this path further? I can add
a test if you could point me in the right direction, but I don't really have
time to prepare documentation on a speculative basis, i.e. if there is a chance
this PR could still be declined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
