[
https://issues.apache.org/jira/browse/BEAM-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262796#comment-17262796
]
Beam JIRA Bot commented on BEAM-11208:
--------------------------------------
This issue is assigned but has not received an update in 30 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> BigQuery storage streams fail with QUOTA_EXCEEDED errors in split
> -----------------------------------------------------------------
>
> Key: BEAM-11208
> URL: https://issues.apache.org/jira/browse/BEAM-11208
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.25.0
> Reporter: Kenneth Jung
> Assignee: Vachan Shetty
> Priority: P2
> Labels: stale-assigned
> Original Estimate: 168h
> Time Spent: 1.5h
> Remaining Estimate: 166.5h
>
> The BigQueryStorageStreamSource attempts to call
> [SplitReadStream](https://cloud.google.com/bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1#bigqueryread)
> for each splitAtFraction call. However, the storage API
> [limits](https://cloud.google.com/bigquery/quotas#storage-limits) the number
> of control plane operations per minute for a given project, which can lead to
> the service being overloaded in the event of a large pipeline. The stream
> reader should not attempt to split a stream once it has learned that the
> stream can no longer be split (e.g. once a SplitReadStream call returns an
> empty response).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
