justinjoseph89 opened a new issue, #6588:
URL: https://github.com/apache/opendal/issues/6588
### Feature Description
Add support for HTTP 429 "Too Many Requests" rate limiting errors in
OpenDAL's S3 service to improve compatibility with few S3-compatible storage
like Oracle Cloud Infrastructure .
### Problem and Solution
OpenDAL's S3 service currently only handles AWS S3's rate limiting pattern
(**HTTP 503** with "**SlowDown**" error code). Many S3-compatible services like
Oracle Cloud Infrastructure Object Storage, Scaleway, and others return HTTP
429 with "**TooManyRequests**" error codes rather than logging SlowDown.
When these services return **HTTP 429**, OpenDAL classifies them as
non-retryable `ErrorKind::Unexpected` errors, causing applications to fail
permanently instead of implementing proper retry logic. This breaks rate
limiting functionality for users of these S3-compatible services.
> 2025-09-23T14:22:32.748030Z ERROR opendal_load_test: read/stat error:
error=Unexpected (permanent) at read, context: { uri:
https://{$namespace}.compat.objectstorage.{$region}.oraclecloud.com/{$objectstore}/{$prefix}/125/667837.data,
response: Parts { status: 429, version: HTTP/1.1, headers:
{"x-amz-request-id": "ams-1:NAkuDFmfxxxxxxxxxxxxxxxxxX1HQ8VQ3", "content-type":
"application/xml; charset=utf-8", "content-length": "119", "date": "Tue, 23 Sep
2025 14:22:32 GMT", "opc-request-id":
"ams-1:NAkuDFmfxxxxxxxxxxxxxxxxxX1HQ8VQ3", "x-api-id": "s3-compatible",
"x-content-type-options": "nosniff", "strict-transport-security":
"max-age=31536000; includeSubDomains", "access-control-allow-origin": "*",
"access-control-allow-methods": "POST,PUT,GET,HEAD,DELETE,OPTIONS",
"access-control-allow-credentials": "true", "access-control-expose-headers":
"access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,content-type,date,opc-client-info,opc-
request-id,strict-transport-security,x-amz-request-id,x-api-id,x-content-type-options"}
}, service: s3, path: {$prefix}/125/667837.data, range: 0-15 } => S3Error {
code: "TooManyRequests", message: "rate-limit exceeded", resource: "",
request_id: "" } key={$prefix}/125/667837.data
### Additional Context
_No response_
### Are you willing to contribute to the development of this feature?
- [x] Yes, I am willing to contribute to the development of this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
