ajdub508 commented on code in PR #28091:
URL: https://github.com/apache/beam/pull/28091#discussion_r1327051890
##########
sdks/python/apache_beam/io/gcp/bigquery_tools.py:
##########
@@ -1491,7 +1493,29 @@ class RetryStrategy(object):
RETRY_NEVER = 'RETRY_NEVER'
RETRY_ON_TRANSIENT_ERROR = 'RETRY_ON_TRANSIENT_ERROR'
- _NON_TRANSIENT_ERRORS = {'invalid', 'invalidQuery', 'notImplemented'}
+ # Values below may be found in reasons provided either in an
+ # error returned by a client method or by an http response as
+ # defined in google.api_core.exceptions
+ _NON_TRANSIENT_ERRORS = {
+ 'invalid',
+ 'invalidQuery',
+ 'notImplemented',
+ 'Moved Permanently',
+ 'Not Modified',
+ 'Temporary Redirect',
+ 'Resume Incomplete',
+ 'Bad Request',
+ 'Unauthorized',
+ 'Forbidden',
+ 'Not Found',
+ 'Method Not Allowed',
+ 'Conflict',
+ 'Length Required',
+ 'Precondition Failed',
+ 'Not Implemented',
+ 'Bad Gateway',
+ 'Gateway Timeout',
+ }
Review Comment:
> Thanks for adding this comprehensive list! I'm a little unfamiliar with a
few of these so will need another pair of eyes to check.
>
> From a cursory search though, it appears "bad gateway" may be a transient
error? Same with "gateway timeout". I would defer to
https://cloud.google.com/bigquery/docs/error-messages for a good list to take
from.
I see what you mean about some of the one I added possibly being transient,
we certainly could take some of the 5XX errors out of that list to broaden it a
bit.
I went narrow, initially, based on the narrow set of api core exceptions
considered transient in api core retry
[here](https://github.com/googleapis/python-api-core/blob/main/google/api_core/retry.py#L125-L128),
where it's only 429 Too Many Requests, 500 Internal Server Error, and 503
Service Unavailable that are considered transient. Other than those 3, all
remaining 4XX, 5XX, and 3XX errors are not considered transient.
Let me know if you think we should remove some 5XX exceptions from the list
or if we should stick with just retrying 429, 500, and 503.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
