[
https://issues.apache.org/jira/browse/AIRFLOW-667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771140#comment-15771140
]
ASF subversion and git services commented on AIRFLOW-667:
---------------------------------------------------------
Commit edf55cd21e1d220c1d19031d7f1b8e666aad770c in incubator-airflow's branch
refs/heads/master from [~criccomini]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=edf55cd ]
[AIRFLOW-712] Fix AIRFLOW-667 to use proper HTTP error properties
Closes #1955 from criccomini/AIRFLOW-712
> Handle 503 errors on google bigquery_hook
> ------------------------------------------
>
> Key: AIRFLOW-667
> URL: https://issues.apache.org/jira/browse/AIRFLOW-667
> Project: Apache Airflow
> Issue Type: Improvement
> Components: gcp
> Affects Versions: Airflow 2.0, Airflow 1.7.1, Airflow 1.7.0
> Reporter: Krishnaveni Mettu
> Assignee: Krishnaveni Mettu
> Priority: Minor
> Fix For: Airflow 1.8
>
>
> Getting 503 errors occasionally while querying google bigquery job status
> Here is the error message from logs -
> [2016-11-30 11:11:46,125] {bigquery_hook.py:454} INFO - Waiting for job to
> complete: xxxxxx, job_xxx_xxxx
> [2016-11-30 11:11:51,131] {discovery.py:838} INFO - URL being requested: GET
> https://www.googleapis.com/bigquery/v2/projects/xxxx/jobs/job_xxxxxx?alt=json
> [2016-11-30 11:12:55,170] {models.py:1321} ERROR - <HttpError 503 when
> requesting
> https://www.googleapis.com/bigquery/v2/projects/xxxx/jobs/job_xxxxxxxx?alt=json
> returned "Error encountered during execution. Retrying may solve the
> problem.">
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1280, in run
> result = task_copy.execute(context=context)
> File
> "/usr/lib/python2.7/site-packages/airflow/contrib/operators/gcs_to_bq.py",
> line 129, in execute
> num_retries=self.num_retries)
> File
> "/usr/lib/python2.7/site-packages/airflow/contrib/hooks/bigquery_hook.py",
> line 403, in run_load
> return self.run_with_configuration(configuration)
> File
> "/usr/lib/python2.7/site-packages/airflow/contrib/hooks/bigquery_hook.py",
> line 456, in run_with_configuration
> job = jobs.get(projectId=self.project_id, jobId=job_id).execute()
> File "/usr/lib/python2.7/site-packages/oauth2client/util.py", line 135, in
> positional_wrapper
> return wrapped(*args, **kwargs)
> File "/usr/lib/python2.7/site-packages/googleapiclient/http.py", line 838,
> in execute
> raise HttpError(resp, content, uri=self.uri)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
