[ https://issues.apache.org/jira/browse/AIRFLOW-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
holdenk reassigned AIRFLOW-1219: -------------------------------- Assignee: (was: holdenk) > Spark submit hook fix realtime logging. > --------------------------------------- > > Key: AIRFLOW-1219 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1219 > Project: Apache Airflow > Issue Type: Bug > Reporter: Kris Geusebroek > Priority: Major > > Redirecting the logging to airflow doesn't work correctly. > Spark's logging goes to stderr and output of the spark program goes to > stdout. In the loop of the two sources stdout is used first. > It seems to be blocking until output is written at the end of the > spark-submit execution. > Fix this by connecting stderr to stdout in the subprocess invocation and only > pipe stdout to the airflow logging -- This message was sent by Atlassian JIRA (v7.6.3#76005)