[
https://issues.apache.org/jira/browse/AIRFLOW-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675427#comment-16675427
]
Ash Berlin-Taylor commented on AIRFLOW-3279:
--------------------------------------------
Those docs are a bit out of date -we've fixed them for what will become the
next release - you can check them out here:
https://github.com/apache/incubator-airflow/blob/master/docs/howto/write-logs.rst#writing-logs-to-google-cloud-storage
> Documentation for Google Logging unclear
> ----------------------------------------
>
> Key: AIRFLOW-3279
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3279
> Project: Apache Airflow
> Issue Type: Bug
> Components: configuration, Documentation, gcp, logging
> Reporter: Paul Velthuis
> Priority: Blocker
>
> The documentation of how to install logging to a Google Cloud bucket is
> unclear.
> I am now following the tutorial on the airflow page:
> [https://airflow.apache.org/howto/write-logs.html]
> Here I find it unclear what part of the 'logger' I have to adjust in the
> `{{airflow/config_templates/airflow_local_settings.py}}`.
>
> The adjustment states:
>
> # Update the airflow.task and airflow.tas_runner blocks to be 'gcs.task'
> instead of 'file.task'. 'loggers':
> Unknown macro: \{ 'airflow.task'}
>
> However what I find in the template is:
> |'loggers': \{\| \|'airflow.processor': { \\\\\\\\| \\\\\\\\|'handlers':
> ['processor'], \\\\\\\\| \\\\\\\\|'level': LOG_LEVEL, \\\\\\\\|
> \\\\\\\\|'propagate': False, \\\\\\\\| \\\\\\\\|},|
> |'airflow.task': {
> \|
> \|'handlers': ['task'],
> \|
> \|'level': LOG_LEVEL,
> \|
> \|'propagate': False,
> \|
> \|},|
> |'flask_appbuilder': {
> \|
> \|'handler': ['console'],
> \|
> \|'level': FAB_LOG_LEVEL,
> \|
> \|'propagate': True,
> \|
> \|}|
> },
>
> Since for me it is very important to do it right at the first time I hope
> some clarity can be provided in what has to be adjusted in the logger. Is it
> only the 'airflow.task' or more?
> Furthermore, at step 6 it is a little unclear what remote_log_conn_id means.
> I would propose to add a little more information to make this more clear.
>
> The current error I am facing is:
> Traceback (most recent call last):
> File "/usr/local/bin/airflow", line 16, in <module>
> from airflow import configuration
> File "/usr/local/lib/python2.7/site-packages/airflow/__init__.py", line 31,
> in <module>
> from airflow import settings
> File "/usr/local/lib/python2.7/site-packages/airflow/settings.py", line 198,
> in <module>
> configure_logging()
> File "/usr/local/lib/python2.7/site-packages/airflow/logging_config.py",
> line 71, in configure_logging
> dictConfig(logging_config)
> File "/usr/local/lib/python2.7/logging/config.py", line 794, in dictConfig
> dictConfigClass(config).configure()
> File "/usr/local/lib/python2.7/logging/config.py", line 568, in configure
> handler = self.configure_handler(handlers[name])
> File "/usr/local/lib/python2.7/logging/config.py", line 733, in
> configure_handler
> result = factory(**kwargs)
> File
> "/usr/local/lib/python2.7/site-packages/airflow/utils/log/gcs_task_handler.py",
> line 30, in __init__
> super(GCSTaskHandler, self).__init__(base_log_folder, filename_template)
> File
> "/usr/local/lib/python2.7/site-packages/airflow/utils/log/file_task_handler.py",
> line 46, in __init__
> self.filename_jinja_template = Template(self.filename_template)
> File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line
> 926, in __new__
> return env.from_string(source, template_class=cls)
> File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line
> 862, in from_string
> return cls.from_code(self, self.compile(source), globals, None)
> File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line
> 565, in compile
> self.handle_exception(exc_info, source_hint=source_hint)
> File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line
> 754, in handle_exception
> reraise(exc_type, exc_value, tb)
> File "<unknown>", line 1, in template
> jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'
> Error in atexit._run_exitfuncs:
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
> func(*targs, **kargs)
> File "/usr/local/lib/python2.7/logging/__init__.py", line 1676, in shutdown
> h.close()
> File
> "/usr/local/lib/python2.7/site-packages/airflow/utils/log/gcs_task_handler.py",
> line 73, in close
> if self.closed:
> AttributeError: 'GCSTaskHandler' object has no attribute 'closed'
> Error in sys.exitfunc:
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
> func(*targs, **kargs)
> File "/usr/local/lib/python2.7/logging/__init__.py", line 1676, in shutdown
> h.close()
> File
> "/usr/local/lib/python2.7/site-packages/airflow/utils/log/gcs_task_handler.py",
> line 73, in close
> if self.closed:
> AttributeError: 'GCSTaskHandler' object has no attribute 'closed'
> If I look at the Airflow code I see the following code for the
> GcsTaskHandler:
> https://github.com/apache/incubator-airflow/blob/v1-10-stable/airflow/utils/log/gcs_task_handler.py
> Here the closed attributed indeed refers to nowhere, does somebody know how
> to resolve this issue?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
