[
https://issues.apache.org/jira/browse/AIRFLOW-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960340#comment-16960340
]
ASF subversion and git services commented on AIRFLOW-5663:
----------------------------------------------------------
Commit 5dd1dbc32207b7e0aeef52eb2c4a6f7a51ae4c27 in airflow's branch
refs/heads/master from wjiangqc
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=5dd1dbc ]
[AIRFLOW-5663] Switch to real-time logging in PythonVirtualenvOperator (#6389)
Co-Authored-By: Kamil Breguła <[email protected]>
> PythonVirtualenvOperator doesn't print logs until the operator is finished
> --------------------------------------------------------------------------
>
> Key: AIRFLOW-5663
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5663
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.10.5
> Reporter: Wei Jiang
> Priority: Major
>
> It seems the {{PythonVirtualenvOperator }}doesn't print out the log until the
> job is done. When we use logging in {{PythonOperator}}, it does print out
> logging in real time. It would be nice for the {{PythonVirtualenvOperator}}
> to have the same functionality as the {{PythonOperator}} so we can see the
> logging output as the PythonVirtualenvOperator makes progress.
> [https://github.com/apache/airflow/blob/master/airflow/operators/python_operator.py#L332]
> {code:python}
> def _execute_in_subprocess(self, cmd):
> try:
> self.log.info("Executing cmd\n%s", cmd)
> output = subprocess.check_output(cmd,
> stderr=subprocess.STDOUT,
> close_fds=True)
> if output:
> self.log.info("Got output\n%s", output)
> except subprocess.CalledProcessError as e:
> self.log.info("Got error output\n%s", e.output)
> raise
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
