hammerhead commented on issue #24014:
URL: https://github.com/apache/airflow/issues/24014#issuecomment-1162819415
Thanks, @josh-fell, the example you provided does work.
However, I couldn't apply it successfully to our implementation, in which
the value of `parameters` is provided by another task:
```python
@task
def parameter_values():
# in our original implementation, this is a database query using
pg_hook
return [
{"table_fqn": "raw_metrics", "column": "ts_day", "value": "v1"},
]
sql = """
DELETE FROM {{ task.mapped_kwargs.parameters[ti.map_index].table_fqn }}
WHERE {{ task.mapped_kwargs.parameters[ti.map_index].column }} = {{
task.mapped_kwargs.parameters[ti.map_index].value }};"""
PostgresOperator.partial(
task_id="delete_partitions",
postgres_conn_id="cratedb_connection",
sql=sql,
).expand(parameters=parameter_values())
```
This fails with an error related to XCom lookups:
```
[2022-06-22, 07:54:34 UTC] {taskinstance.py:1889} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line
1451, in _run_raw_task
self._execute_task_with_callbacks(context, test_mode)
File
"/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line
1555, in _execute_task_with_callbacks
task_orig = self.render_templates(context=context)
File
"/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line
2212, in render_templates
rendered_task = self.task.render_template_fields(context)
File
"/usr/local/lib/python3.9/site-packages/airflow/models/mappedoperator.py", line
726, in render_template_fields
self._do_render_template_fields(
File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py",
line 68, in wrapper
return func(*args, **kwargs)
File
"/usr/local/lib/python3.9/site-packages/airflow/models/abstractoperator.py",
line 344, in _do_render_template_fields
rendered_content = self.render_template(
File
"/usr/local/lib/python3.9/site-packages/airflow/models/abstractoperator.py",
line 391, in render_template
return render_template_to_string(template, context)
File "/usr/local/lib/python3.9/site-packages/airflow/utils/helpers.py",
line 296, in render_template_to_string
return render_template(template, context, native=False)
File "/usr/local/lib/python3.9/site-packages/airflow/utils/helpers.py",
line 291, in render_template
return "".join(nodes)
File "<template>", line 14, in root
File "/usr/local/lib/python3.9/site-packages/jinja2/sandbox.py", line 303,
in getitem
return obj[argument]
File "/usr/local/lib/python3.9/site-packages/airflow/models/xcom_arg.py",
line 77, in __getitem__
raise ValueError(f"XComArg only supports str lookup, received
{type(item).__name__}")
ValueError: XComArg only supports str lookup, received int
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
