ahmarsuhail commented on PR #4706: URL: https://github.com/apache/hadoop/pull/4706#issuecomment-1227250089
Thanks @dannycjones, have moved those things to constants. RE using `eu-west-1` and the region logic in general: Not using `us-east-1` as `headBucket()` fails with that region. This is because `us-east-1` uses the endpoint `s3.amazonaws.com`, which resolves `bucket.s3.amazonaws.com` to the actual region the bucket is in. As the request is signed with us-east-1 and not the bucket's region, it fails. For more info, see [this](https://github.com/aws/aws-sdk-java/issues/1338) issue. I'm not sure how the region logic should behave and how it should handle failures. Returning `us-east-1` at the end [here](https://github.com/apache/hadoop/pull/4706/files#diff-a8dabc9bdb3ac3b04f92eadd1e3d9a7076d8983ca4fb7d1d146a1ac725caa309R558) is not of much use, as without cross region access, if the region is configured incorrectly, any request to S3 will fail. Instead, to handle network failures etc, we should probably add some retry logic in [this](https://github.com/apache/hadoop/pull/4706/files#diff-a8dabc9bdb3ac3b04f92eadd1e3d9a7076d8983ca4fb7d1d146a1ac725caa309R530) method. I'm also not sure if this new region/endpoint logic is sufficient to handle third party stores, keen to know what other people think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
