Szymon Bilinski created AIRFLOW-3069:
----------------------------------------
Summary: Decode output of S3 file transform operator
Key: AIRFLOW-3069
URL: https://issues.apache.org/jira/browse/AIRFLOW-3069
Project: Apache Airflow
Issue Type: Improvement
Components: aws
Affects Versions: 1.10.0
Reporter: Szymon Bilinski
Assignee: Szymon Bilinski
h3. Current behaviour
{{S3FileTransformOperator}} logs {{stdout}} of the underlying process as such:
{code}
[2018-09-15 23:17:13,850] {{s3_file_transform_operator.py:122}} INFO -
Transform script stdout b'Copying /tmp/tmpd5rjo8g0 to /tmp/tmpd3vkhzte\nDone\n'
{code}
While {{stderr}} is omitted entirely, unless exit code is not {{0}} (in this
case it's included in the exception message only).
h3. Proposed behaviour
1. Both streams are logged, regardless of the underlying process outcome (i.e.
success or failure).
2. Stream output is decoded before logging (e.g. {{\n}} is replaced with an
actual new line).
3. If {{transform_script}} fails, the exception message contains return code of
the process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
