Kevin Peterson created BEAM-5514:
------------------------------------
Summary: BigQueryIO doesn't handle quotaExceeded errors properly
Key: BEAM-5514
URL: https://issues.apache.org/jira/browse/BEAM-5514
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Reporter: Kevin Peterson
Assignee: Chamikara Jayalath
When exceeding a streaming quota for BigQuery insertAll requests, BigQuery
returns a 403 with reason "quotaExceeded".
The current implementation of BigQueryIO does not consider this to be a rate
limited exception, and therefore does not perform exponential backoff properly,
leading to repeated calls to BQ.
The actual error is in the
[ApiErrorExtractor|https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L739]
class, which is called from
[BigQueryServicesImpl|https://github.com/GoogleCloudPlatform/bigdata-interop/blob/master/util/src/main/java/com/google/cloud/hadoop/util/ApiErrorExtractor.java#L263]
to determine how to retry the failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
