[
https://issues.apache.org/jira/browse/BEAM-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16866075#comment-16866075
]
Heejong Lee commented on BEAM-7424:
-----------------------------------
Per user rate limit exceeded error is 403, application-level rate limit
exceeded error is 429. Maybe we can use the same method
[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/utils/retry.py#L114]
for handling 403 and 429.
> Retry HTTP 429 errors from GCS w/ exponential backoff when reading data
> -----------------------------------------------------------------------
>
> Key: BEAM-7424
> URL: https://issues.apache.org/jira/browse/BEAM-7424
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp, io-python-gcp, sdk-py-core
> Reporter: Chamikara Jayalath
> Assignee: Heejong Lee
> Priority: Blocker
> Fix For: 2.14.0
>
>
> This has to be done for both Java and Python SDKs.
> Seems like Java SDK already retries 429 errors w/o backoff (please verify):
> [https://github.com/apache/beam/blob/master/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/RetryHttpRequestInitializer.java#L185]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
