Wei Jiang created AIRFLOW-5663:
----------------------------------
Summary: PythonVirtualenvOperator doesn't print logs until the
operator is finished
Key: AIRFLOW-5663
URL: https://issues.apache.org/jira/browse/AIRFLOW-5663
Project: Apache Airflow
Issue Type: Bug
Components: operators
Affects Versions: 1.10.5
Reporter: Wei Jiang
It seems the {{PythonVirtualenvOperator }}doesn't print out the log until the
job is done. When we use logging in {{PythonOperator}}, it does print out
logging in real time. It would be nice for the {{PythonVirtualenvOperator}} to
have the same functionality as the {{PythonOperator}} so we can see the logging
output as the PythonVirtualenvOperator makes progress.
[https://github.com/apache/airflow/blob/master/airflow/operators/python_operator.py#L332]
{code:python}
def _execute_in_subprocess(self, cmd):
try:
self.log.info("Executing cmd\n%s", cmd)
output = subprocess.check_output(cmd,
stderr=subprocess.STDOUT,
close_fds=True)
if output:
self.log.info("Got output\n%s", output)
except subprocess.CalledProcessError as e:
self.log.info("Got error output\n%s", e.output)
raise
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
