Krishnaveni Mettu created AIRFLOW-666:
-----------------------------------------
Summary: Handle 503 errors on google bigquery_hook
Key: AIRFLOW-666
URL: https://issues.apache.org/jira/browse/AIRFLOW-666
Project: Apache Airflow
Issue Type: Improvement
Reporter: Krishnaveni Mettu
Priority: Minor
Getting 503 errors occasionally while querying google bigquery job status
Here is the error message from logs -
[2016-11-30 11:11:46,125] {bigquery_hook.py:454} INFO - Waiting for job to
complete: xxxxxx, job_xxx_xxxx
[2016-11-30 11:11:51,131] {discovery.py:838} INFO - URL being requested: GET
https://www.googleapis.com/bigquery/v2/projects/xxxx/jobs/job_xxxxxx?alt=json
[2016-11-30 11:12:55,170] {models.py:1321} ERROR - <HttpError 503 when
requesting
https://www.googleapis.com/bigquery/v2/projects/xxxx/jobs/job_xxxxxxxx?alt=json
returned "Error encountered during execution. Retrying may solve the problem.">
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1280, in run
result = task_copy.execute(context=context)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/operators/gcs_to_bq.py", line
129, in execute
num_retries=self.num_retries)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/hooks/bigquery_hook.py", line
403, in run_load
return self.run_with_configuration(configuration)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/hooks/bigquery_hook.py", line
456, in run_with_configuration
job = jobs.get(projectId=self.project_id, jobId=job_id).execute()
File "/usr/lib/python2.7/site-packages/oauth2client/util.py", line 135, in
positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/googleapiclient/http.py", line 838, in
execute
raise HttpError(resp, content, uri=self.uri)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
