Ash Berlin-Taylor created AIRFLOW-4990:
------------------------------------------
Summary: Logs from tasks get "wrapped" twice
Key: AIRFLOW-4990
URL: https://issues.apache.org/jira/browse/AIRFLOW-4990
Project: Apache Airflow
Issue Type: Bug
Components: logging
Affects Versions: 1.9.0
Reporter: Ash Berlin-Taylor
As mentioned in AIRFLOW-1582 (but never opened as a ticket), when a task uses
logging then the messages are formatted twice:
{noformat}
[2017-09-07 00:00:16,305] {base_task_runner.py:95} INFO - Subtask: [2017-09-07
00:00:16,304] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python3.5/lib2to3/Grammar.txt
{noformat}
Note the two timestamps {{{}} are not identical.
The "inner" timestamp ({{[2017-09-07 00:00:16,304]}} to end of line) comes from
the task execution process (the one with {{airflow run ... --raw}} command
line, and the prefix before that comes from the "supervisor" process.
Ideally so this plays well with producing JSON log messages, we should to more
than change the formatting for the actual executor task. Something like
https://docs.python.org/3/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network
might be the way to achieve this.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
