Jorge Alpedrinha Ramos created AIRFLOW-1037:
-----------------------------------------------
Summary: Webserver can't fetch large log files from Remote worker
Key: AIRFLOW-1037
URL: https://issues.apache.org/jira/browse/AIRFLOW-1037
Project: Apache Airflow
Issue Type: Bug
Components: celery, logging, webserver
Affects Versions: Airflow 1.8
Environment: Running airflow with docker using CeleryExecutor
Reporter: Jorge Alpedrinha Ramos
When tasks logs are too big, the webserver isn't able to fetch them. This may
be related with https://github.com/kennethreitz/requests/issues/2422.
I've recovered a stacktrace from the worker:
{code}
worker_1 | [2017-03-24 10:29:39,574] {_internal.py:87} INFO -
172.18.0.8 - - [24/Mar/2017 10:29:39] "GET
/log/rates.ticker-to-analytics/parse-syslog/2017-01-12T07:45:00 HTTP/1.1" 200 -
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/bin/airflow", line 28, in <module>
worker_1 | args.func(args)
worker_1 | File
"/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 858, in
serve_logs
worker_1 | host='0.0.0.0', port=WORKER_LOG_SERVER_PORT)
worker_1 | File
"/usr/local/lib/python2.7/site-packages/flask/app.py", line 843, in run
worker_1 | run_simple(host, port, self, **options)
worker_1 | File
"/usr/local/lib/python2.7/site-packages/werkzeug/serving.py", line 736, in
run_simple
worker_1 | inner()
worker_1 | File
"/usr/local/lib/python2.7/site-packages/werkzeug/serving.py", line 699, in inner
worker_1 | srv.serve_forever()
worker_1 | File
"/usr/local/lib/python2.7/site-packages/werkzeug/serving.py", line 536, in
serve_forever
worker_1 | HTTPServer.serve_forever(self)
worker_1 | File "/usr/local/lib/python2.7/SocketServer.py", line 233,
in serve_forever
worker_1 | self._handle_request_noblock()
worker_1 | File "/usr/local/lib/python2.7/SocketServer.py", line 292,
in _handle_request_noblock
worker_1 | self.handle_error(request, client_address)
worker_1 | File "/usr/local/lib/python2.7/SocketServer.py", line 290,
in _handle_request_noblock
worker_1 | self.process_request(request, client_address)
worker_1 | File "/usr/local/lib/python2.7/SocketServer.py", line 318,
in process_request
worker_1 | self.finish_request(request, client_address)
worker_1 | File "/usr/local/lib/python2.7/SocketServer.py", line 331,
in finish_request
worker_1 | self.RequestHandlerClass(request, client_address, self)
worker_1 | File "/usr/local/lib/python2.7/SocketServer.py", line 654,
in __init__
worker_1 | self.finish()
worker_1 | File "/usr/local/lib/python2.7/SocketServer.py", line 713,
in finish
worker_1 | self.wfile.close()
worker_1 | File "/usr/local/lib/python2.7/socket.py", line 283, in
close
worker_1 | self.flush()
worker_1 | File "/usr/local/lib/python2.7/socket.py", line 307, in
flush
worker_1 |
self._sock.sendall(view[write_offset:write_offset+buffer_size])
worker_1 | socket.error: [Errno 32] Broken pipe
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
