[
https://issues.apache.org/jira/browse/AIRFLOW-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Podlepaev updated AIRFLOW-2774:
--------------------------------------
Description:
**DataFlowPythonOperator needs to support DirectRunner as a runner option to
facilitate local end-to-end testing of the entire Airflow dag. Right now if
DirectRunner is set via job options, the DataFlowHook will wait infinitely
trying to get status of the remote job which does not exist:
_DataflowJob(self.get_conn(), variables['project'], name,
variables['region'], self.poll_sleep).wait_for_done()
was:
**DataFlowPythonOperator needs to support DirectRunner as a runner option to
facilitate local testing of the entire pipeline. Right now if DirectRunner is
set via job options, the DataFlowHook will wait infinitely trying to get status
of the remote job which does not exist:
_DataflowJob(self.get_conn(), variables['project'], name,
variables['region'], self.poll_sleep).wait_for_done()
> DataFlowPythonOperator needs to support DirectRunner to speed up end-to-end
> testing of Airflow dag
> --------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-2774
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2774
> Project: Apache Airflow
> Issue Type: Improvement
> Components: Dataflow
> Affects Versions: 1.9.0
> Reporter: Evgeny Podlepaev
> Priority: Minor
>
> **DataFlowPythonOperator needs to support DirectRunner as a runner option to
> facilitate local end-to-end testing of the entire Airflow dag. Right now if
> DirectRunner is set via job options, the DataFlowHook will wait infinitely
> trying to get status of the remote job which does not exist:
> _DataflowJob(self.get_conn(), variables['project'], name,
> variables['region'], self.poll_sleep).wait_for_done()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
