[ https://issues.apache.org/jira/browse/AIRFLOW-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Velthuis closed AIRFLOW-3279. ---------------------------------- Assignee: Paul Velthuis > Documentation for Google Logging unclear > ---------------------------------------- > > Key: AIRFLOW-3279 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3279 > Project: Apache Airflow > Issue Type: Bug > Components: configuration, Documentation, gcp, logging > Reporter: Paul Velthuis > Assignee: Paul Velthuis > Priority: Blocker > > The documentation of how to install logging to a Google Cloud bucket is > unclear. > I am now following the tutorial on the airflow page: > [https://airflow.apache.org/howto/write-logs.html] > Here I find it unclear what part of the 'logger' I have to adjust in the > `{{airflow/config_templates/airflow_local_settings.py}}`. > > The adjustment states: > > # Update the airflow.task and airflow.tas_runner blocks to be 'gcs.task' > instead of 'file.task'. 'loggers': > Unknown macro: \{ 'airflow.task'} > > However what I find in the template is: > |'loggers': \{\| \|'airflow.processor': { \\\\\\\\| \\\\\\\\|'handlers': > ['processor'], \\\\\\\\| \\\\\\\\|'level': LOG_LEVEL, \\\\\\\\| > \\\\\\\\|'propagate': False, \\\\\\\\| \\\\\\\\|},| > |'airflow.task': { > \| > \|'handlers': ['task'], > \| > \|'level': LOG_LEVEL, > \| > \|'propagate': False, > \| > \|},| > |'flask_appbuilder': { > \| > \|'handler': ['console'], > \| > \|'level': FAB_LOG_LEVEL, > \| > \|'propagate': True, > \| > \|}| > }, > > Since for me it is very important to do it right at the first time I hope > some clarity can be provided in what has to be adjusted in the logger. Is it > only the 'airflow.task' or more? > Furthermore, at step 6 it is a little unclear what remote_log_conn_id means. > I would propose to add a little more information to make this more clear. > > The current error I am facing is: > Traceback (most recent call last): > File "/usr/local/bin/airflow", line 16, in <module> > from airflow import configuration > File "/usr/local/lib/python2.7/site-packages/airflow/__init__.py", line 31, > in <module> > from airflow import settings > File "/usr/local/lib/python2.7/site-packages/airflow/settings.py", line 198, > in <module> > configure_logging() > File "/usr/local/lib/python2.7/site-packages/airflow/logging_config.py", > line 71, in configure_logging > dictConfig(logging_config) > File "/usr/local/lib/python2.7/logging/config.py", line 794, in dictConfig > dictConfigClass(config).configure() > File "/usr/local/lib/python2.7/logging/config.py", line 568, in configure > handler = self.configure_handler(handlers[name]) > File "/usr/local/lib/python2.7/logging/config.py", line 733, in > configure_handler > result = factory(**kwargs) > File > "/usr/local/lib/python2.7/site-packages/airflow/utils/log/gcs_task_handler.py", > line 30, in __init__ > super(GCSTaskHandler, self).__init__(base_log_folder, filename_template) > File > "/usr/local/lib/python2.7/site-packages/airflow/utils/log/file_task_handler.py", > line 46, in __init__ > self.filename_jinja_template = Template(self.filename_template) > File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line > 926, in __new__ > return env.from_string(source, template_class=cls) > File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line > 862, in from_string > return cls.from_code(self, self.compile(source), globals, None) > File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line > 565, in compile > self.handle_exception(exc_info, source_hint=source_hint) > File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line > 754, in handle_exception > reraise(exc_type, exc_value, tb) > File "<unknown>", line 1, in template > jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}' > Error in atexit._run_exitfuncs: > Traceback (most recent call last): > File "/usr/local/lib/python2.7/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/local/lib/python2.7/logging/__init__.py", line 1676, in shutdown > h.close() > File > "/usr/local/lib/python2.7/site-packages/airflow/utils/log/gcs_task_handler.py", > line 73, in close > if self.closed: > AttributeError: 'GCSTaskHandler' object has no attribute 'closed' > Error in sys.exitfunc: > Traceback (most recent call last): > File "/usr/local/lib/python2.7/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/local/lib/python2.7/logging/__init__.py", line 1676, in shutdown > h.close() > File > "/usr/local/lib/python2.7/site-packages/airflow/utils/log/gcs_task_handler.py", > line 73, in close > if self.closed: > AttributeError: 'GCSTaskHandler' object has no attribute 'closed' > If I look at the Airflow code I see the following code for the > GcsTaskHandler: > https://github.com/apache/incubator-airflow/blob/v1-10-stable/airflow/utils/log/gcs_task_handler.py > Here the closed attributed indeed refers to nowhere, does somebody know how > to resolve this issue? > -- This message was sent by Atlassian JIRA (v7.6.3#76005)