dmateusp opened a new issue, #24169: URL: https://github.com/apache/beam/issues/24169
### What would you like to happen? Beam version: `2.36.0` Jobs reading from the BigQuery Storage API can hit Quota limits and eventually fail with `The worker lost contact with the service.` (Dataflow runner) The reason for the failure can be diagnosed on the API Monitoring (GCP) for BigQuery Storage API calls, where a spike in `429`s can be seen for `ReadRows` when the throughput of the job starts going downhill. On the GCP side we've tried to resolve it through Quota increases, but it would be helpful to de-risk jobs crashing related to: * https://github.com/apache/beam/pull/15445 <- Shouldn't this PR have pervented this kind of issue? ### Issue Priority Priority: 2 ### Issue Component Component: io-java-gcp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
