[
https://issues.apache.org/jira/browse/BEAM-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305510#comment-17305510
]
Beam JIRA Bot commented on BEAM-11208:
--------------------------------------
This issue is P2 but has been unassigned without any comment for 60 days so it
has been labeled "stale-P2". If this issue is still affecting you, we care!
Please comment and remove the label. Otherwise, in 14 days the issue will be
moved to P3.
Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed
explanation of what these priorities mean.
> BigQuery storage streams fail with QUOTA_EXCEEDED errors in split
> -----------------------------------------------------------------
>
> Key: BEAM-11208
> URL: https://issues.apache.org/jira/browse/BEAM-11208
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.25.0
> Reporter: Kenneth Jung
> Priority: P2
> Labels: stale-P2
> Original Estimate: 168h
> Time Spent: 1.5h
> Remaining Estimate: 166.5h
>
> The BigQueryStorageStreamSource attempts to call
> [SplitReadStream](https://cloud.google.com/bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1#bigqueryread)
> for each splitAtFraction call. However, the storage API
> [limits](https://cloud.google.com/bigquery/quotas#storage-limits) the number
> of control plane operations per minute for a given project, which can lead to
> the service being overloaded in the event of a large pipeline. The stream
> reader should not attempt to split a stream once it has learned that the
> stream can no longer be split (e.g. once a SplitReadStream call returns an
> empty response).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
