[
https://issues.apache.org/jira/browse/AIRFLOW-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873304#comment-16873304
]
Galak commented on AIRFLOW-4833:
--------------------------------
I had the same issue using Airflow 1.10.2: a trailing newline on a template
field value is removed.
I dug into the code and discovered this is due to the way a {{Dag}} is
instantiating a jinja2 environment for template rendering:
{code}
def get_template_env(self):
"""
Returns a jinja2 Environment while taking into account the DAGs
template_searchpath, user_defined_macros and user_defined_filters
"""
searchpath = [self.folder]
if self.template_searchpath:
searchpath += self.template_searchpath
env = jinja2.Environment(
loader=jinja2.FileSystemLoader(searchpath),
undefined=self.template_undefined,
extensions=["jinja2.ext.do"],
cache_size=0)
if self.user_defined_macros:
env.globals.update(self.user_defined_macros)
if self.user_defined_filters:
env.filters.update(self.user_defined_filters)
return env
{code}
jinja2.Environment has a property {{keep_trailing_newline}} set to False by
default
(see
[https://stackoverflow.com/questions/40832588/jinja2-ignoring-last-new-line]
and [http://jinja.pocoo.org/docs/2.10/api/#jinja2.Environment]).
I would suggest to have a way to add {{jinja2.Environment}} options to a
{{Dag}}. Something like :
{code:java}
DAG(dag_id='my-dag',
templating_options={
'keep_trailing_newline': True,
# some other jinja2 Environment options here
}){code}
What do you think about it ?
> Jinja templating removes newlines
> ---------------------------------
>
> Key: AIRFLOW-4833
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4833
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.10.2
> Reporter: Francesco Macagno
> Priority: Minor
>
> When using an operator that has Jinja templating enabled for a field, if the
> field value ends with a newline then the newline is removed, regardless of
> whether there was a template in the string.
>
> This came up when attempting to send data to Prometheus pushgateway using the
> SimpleHttpOperator. Pushgateway requires a newline at the end of every entry,
> so the removal of the newline at the end of the data parameter causes the
> request to fail in a way that is difficult to debug.
>
> This can be gotten around by including a space after the newline character,
> though this is not a great solution. The space is ignored by pushgateway.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
