Kevin Crouse created ARROW-16826:
------------------------------------
Summary: Possible configuration issue in 2.3.2 on Ubuntu jammy (?)
Key: ARROW-16826
URL: https://issues.apache.org/jira/browse/ARROW-16826
Project: Apache Arrow
Issue Type: Bug
Environment: Airflow 2.3.2
jinja 3.0.3 (also tried jinja 3.1.*)
Ubuntu 22.04, amd64, on AWS
Reporter: Kevin Crouse
This seems odd and there may be another answer for it, but I can't find
anything. We have been running 2.1.4 on Ubuntu Xenial for about a year without
much trouble. So, we are switching over to the new Ubuntu 22.04 LTS environment
with all up to date libraries.
When attempting to run Airflow 2.3.2, I am getting strange config-related
errors when processing our custom config variable (full stack trace at the
bottom).
# ` log_processor_filename_template = \{{{{ filename }}}}.log` - This is the
value included in the default template, but it fails to process with the error
`jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'`
# The default `log_filename_template` value also fails to process with a
similar error
Commenting these out allows the config file to process, however it is not
substituting templates correctly. The multiple `\{AIRFLOW_HOME}` references
are being interpreted literally instead of being substituted for the actual
value. If I change these to reference an environment variable (e.g.,
`${AIRFLOW_HOME}`), it works fine.
Here's the stack trace for the log filename template. I put in a custom
breakpoint to determine that it indeed is just for the log_filename_template
and log_processor_filename_template variables:
```
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Unable to load the config,
contains a configuration error.
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call
last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 565, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: handler =
self.configure_handler(handlers[name])
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 746, in configure_handler
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: result = factory(**kwargs)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/utils/log/file_processor_handler.py",
line 44, in __init__
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: self.filename_template,
self.filename_jinja_template = parse_template_string(filename_template)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/utils/helpers.py", line 176,
in parse_template_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return None,
jinja2.Template(template_string)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 1195, in __new__
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return
env.from_string(source, template_class=cls)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 1092, in
from_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return cls.from_code(self,
self.compile(source), gs, None)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 757, in compile
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]:
self.handle_exception(source=source_hint)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in
handle_exception
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise
rewrite_traceback_stack(source=source)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "<unknown>", line 1, in
template
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]:
jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: The above exception was the
direct cause of the following exception:
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call
last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/bin/airflow", line 5, in <module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: from airflow.__main__
import main
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/__init__.py", line 47, in
<module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: settings.initialize()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 561, in
initialize
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: LOGGING_CLASS_PATH =
configure_logging()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 73,
in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise e
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 68,
in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: dictConfig(logging_config)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 810, in dictConfig
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]:
dictConfigClass(config).configure()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File
"/usr/lib/python3.10/logging/config.py", line 572, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise ValueError('Unable
to configure handler '
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: ValueError: Unable to
configure handler 'processor'
```
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
