xinbinhuang edited a comment on issue #14813:
URL: https://github.com/apache/airflow/issues/14813#issuecomment-820876982
Hi @rbankston, I don't think this is an airflow bug, but just how the
`subprocess` and the `logging` module work. In a word, you need to propagate
back the `stdout` from your subprocess back in order to log in airflow. There
are two options:
1. propagate the `stdout` yourself, i.e.:
```python
...
p = subprocess.run(shlex.split(cmd), capture_output=True)
print(p.stdout)
ret = p.returncode
...
```
In this case, the `p.stdout` is a simple string and you may need to parse it
yourself.
2. Use the `SubprocessHook` (recommended). This will automatically capture
the stdout and log them in airflow for you.
```python
sub_result: SubprocessResult =
SubprocessHook().run_command(shlex.split(cmd))
print(sub_result.exit_code)
# this is only necessary if you want to do extra stuff with the log other
than logging in airflow
print(sub_result.output)
```
cc: @dstandish
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
