Copilot commented on code in PR #12713: URL: https://github.com/apache/gluten/pull/12713#discussion_r3727066462
########## docs/get-started/VeloxS3.md: ########## @@ -71,6 +71,23 @@ These settings apply to all buckets by default. To override a single bucket, use Gluten's static backend pass-through prefix, for example `spark.gluten.velox.hive.s3.bucket.my-bucket.part-upload-async`. +# Known Limitation: Cross-Region S3 Access + +If your S3 data and your configured endpoint are in different AWS regions — for example, your bucket is in `us-west-2` but you have set the endpoint to `us-east-1` (or left it unconfigured, causing requests to default to `us-east-1`) — you will encounter a runtime error. + +Unlike the Java S3 SDK used by `s3a://` (which often handles 301 cross-region redirects automatically), the AWS C++ SDK (`aws-sdk-cpp`) does **not** automatically follow 301 `PermanentRedirect` responses for S3 payload requests. It treats the redirect as a non-retriable error: + +``` +Error Type: SDK Error (100) +Error: PermanentRedirect — The bucket you are attempting to access must be addressed using the specified endpoint. +``` + +**Resolution:** Set the endpoint explicitly to the region where your bucket resides, for example: + +```properties +spark.hadoop.fs.s3a.endpoint=s3.us-west-2.amazonaws.com Review Comment: The endpoint example omits the URL scheme, but earlier in this doc `spark.hadoop.fs.s3a.endpoint` is shown as a full URL (e.g. `https://s3.us-west-1.amazonaws.com`). Using the full URL here avoids ambiguity and keeps the guidance consistent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
