[
https://issues.apache.org/jira/browse/AIRFLOW-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963774#comment-16963774
]
Mariusz Górski commented on AIRFLOW-5774:
-----------------------------------------
[~playermanny2] updated the link, it's already on master but not a part of
latest release. Please see:
[https://github.com/apache/airflow/commit/a6daeb544e815fe350a96d24ae3bb14aee4079a7#diff-5fe5f93b69bf5129e9c10232ee13be30]
> PapermillOperator fails to render parameters
> --------------------------------------------
>
> Key: AIRFLOW-5774
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5774
> Project: Apache Airflow
> Issue Type: Bug
> Components: utils
> Affects Versions: 1.10.4
> Reporter: Mariusz Górski
> Priority: Minor
> Labels: bug
>
> *Problem*
> When running sample PapermillOperator described in docs, it fails with
> {code:java}
> //[2019-10-27 10:48:53,134] {{taskinstance.py:1047}} ERROR - Can't compile
> non template nodes
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line
> 922, in _run_raw_task
> result = task_copy.execute(context=context)
> File
> "/usr/local/lib/python3.7/site-packages/airflow/operators/papermill_operator.py",
> line 71, in execute
> logging.info(inlet.as_dict())
> File "/usr/local/lib/python3.7/site-packages/airflow/lineage/datasets.py",
> line 85, in as_dict
> attributes[key] = env.from_string(value).render(**self.context)
> File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line
> 880, in from_string
> return cls.from_code(self, self.compile(source), globals, None)
> File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line
> 581, in compile
> defer_init=defer_init)
> File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line
> 543, in _generate
> optimized=self.optimized)
> File "/usr/local/lib/python3.7/site-packages/jinja2/compiler.py", line 78,
> in generate
> raise TypeError('Can\'t compile non template nodes')
> TypeError: Can't compile non template nodes
> {code}
> *Reason*
> It's because: _class NoteBook(DataSet)_ expects _parameters_ to be _strings_,
> while papermill _execute_notebook_ function requires _parameters_ to be
> _dicts_. When we pass dict to _NoteBook.parameters_, then jinja templating
> fails when dict is passed to _env.from_string(value).render(**context)_ call.
> *Solution:*
> Handling of nested structs in DataSet class
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
