### Jira

- [x] My PR addresses the following [Airflow 
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references 
them in the PR title.
  - 
[https://jira.apache.org/jira/browse/AIRFLOW-3069](https://jira.apache.org/jira/browse/AIRFLOW-3069)

### Description

- [x] Here are some details about my PR, including screenshots of any UI 
changes:

The output of the process spawned by `S3FileTransformOperator` is not properly 
decoded, which makes reading logs rather difficult. Additionally, the `stderr` 
stream is only shown when process exit code is not equal to `0`.

I would like to propose the following changes to `S3FileTransformOperator`:

- Send both output streams (stdout & stderr) to the logger, regardless of the 
outcome (if any data is present)
- Decode the output, so that new lines can be displayed correctly.
- Include process exit code in the exception message, if the process fails.

### Tests

- [x] My PR adds the following unit tests __OR__ does not need testing for this 
extremely good reason:

I've added a separate case for testing `transform_script` with output present. 
Since logging is essential in this case, the test checks if a valid message was 
passed to the logging module. 

### Commits

- [x] My commits all reference Jira issues in their subject lines, and I have 
squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
  1. Subject is separated from body by a blank line
  1. Subject is limited to 50 characters (not including Jira issue reference)
  1. Subject does not end with a period
  1. Subject uses the imperative mood ("add", not "adding")
  1. Body wraps at 72 characters
  1. Body explains "what" and "why", not "how"

### Documentation

- [x] In case of new functionality, my PR adds documentation that describes how 
to use it.
  - When adding new operators/hooks/sensors, the autoclass documentation 
generation needs to be added.

### Code Quality

- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


[ Full content available at: 
https://github.com/apache/incubator-airflow/pull/3914 ]
This message was relayed via gitbox.apache.org for [email protected]

Reply via email to