Crystal Qian created AIRFLOW-1718:
-------------------------------------
Summary: Increase num_retries polling value on Dataproc hook
Key: AIRFLOW-1718
URL: https://issues.apache.org/jira/browse/AIRFLOW-1718
Project: Apache Airflow
Issue Type: Bug
Reporter: Crystal Qian
Assignee: Crystal Qian
Priority: Minor
Currently, `num_retries = 0` when execute() is called
(https://google.github.io/google-api-python-client/docs/epy/googleapiclient.http.HttpRequest-class.html#execute),
which causes intermittent 500 errors
(https://stackoverflow.com/questions/46522261/deadline-exceeded-when-airflow-runs-spark-jobs).
We should increase this to allow retries for internal Dataproc queries to
other services in the short-term; also seeing if the `num_retries` count can be
increased at the _google-api-python-client_ level in the long-term.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
