Kyle Hamlin created AIRFLOW-2363:
------------------------------------
Summary: S3 remote logging appending tuple instead of str
Key: AIRFLOW-2363
URL: https://issues.apache.org/jira/browse/AIRFLOW-2363
Project: Apache Airflow
Issue Type: Bug
Components: logging
Reporter: Kyle Hamlin
Fix For: 1.10.0
A recent merge into master that added support for Elasticsearch logging seems
to have broken S3 logging by returning a tuple instead of a string.
[https://github.com/apache/incubator-airflow/commit/ec38ba9594395de04ec932481212a86fbe9ae107#diff-0442332ecbe42ebbf426911c68d8cd4aR128]
following errors thrown:
*Session NoneType error*
Traceback (most recent call last):
File
"/usr/local/lib/python3.6/site-packages/airflow/utils/log/s3_task_handler.py",
line 171, in s3_write
encrypt=configuration.conf.getboolean('core', 'ENCRYPT_S3_LOGS'),
File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py", line
274, in load_string
encrypt=encrypt)
File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py", line
313, in load_bytes
client = self.get_conn()
File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py", line
34, in get_conn
return self.get_client_type('s3')
File
"/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/aws_hook.py",
line 151, in get_client_type
session, endpoint_url = self._get_credentials(region_name)
File
"/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/aws_hook.py",
line 97, in _get_credentials
connection_object = self.get_connection(self.aws_conn_id)
File "/usr/local/lib/python3.6/site-packages/airflow/hooks/base_hook.py",
line 82, in get_connection
conn = random.choice(cls.get_connections(conn_id))
File "/usr/local/lib/python3.6/site-packages/airflow/hooks/base_hook.py",
line 77, in get_connections
conns = cls._get_connections_from_db(conn_id)
File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 72,
in wrapper
with create_session() as session:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 41,
in create_session
session = settings.Session()
TypeError: 'NoneType' object is not callable
*TypeError must be str not tuple*
[2018-04-16 18:37:28,200] ERROR in app: Exception on
/admin/airflow/get_logs_with_metadata [GET]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1982, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1614, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1517, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33, in
reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.6/site-packages/flask_admin/base.py", line 69,
in inner
return self._run_view(f, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/flask_admin/base.py", line 368,
in _run_view
return fn(self, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/flask_login.py", line 755, in
decorated_view
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/www/utils.py", line
269, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 74,
in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/www/views.py", line
770, in get_logs_with_metadata
logs, metadatas = handler.read(ti, try_number, metadata=metadata)
File
"/usr/local/lib/python3.6/site-packages/airflow/utils/log/file_task_handler.py",
line 165, in read
logs[i] += log
TypeError: must be str, not tuple
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
