ihji commented on a change in pull request #12403:
URL: https://github.com/apache/beam/pull/12403#discussion_r472507737
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java
##########
@@ -867,6 +872,7 @@ public void deleteDataset(String projectId, String
datasetId)
}
try {
sleeper.sleep(nextBackoffMillis);
+ throttlingMilliSeconds.inc(nextBackoffMillis);
Review comment:
Yes, we should use `backoff1`. Rate limit errors only reach to this
point after 2 minutes of backoffs by `backoff1` silently inside of the future.
Why do you think it's over-calculated? Each thread is doing its own insert job
and it doesn't look strange to me to calculate the total throttling time by
adding all backoff times from parallel threads.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
