[
https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774996#comment-17774996
]
ASF GitHub Bot commented on HADOOP-18908:
-----------------------------------------
steveloughran opened a new pull request, #6187:
URL: https://github.com/apache/hadoop/pull/6187
### Description of PR
#6106 with one extra commit
Removes the region probe to S3 and reinstates the region resolution logic
that was there before the SDK V2 upgrade.
* Region probe is no longer needed as SDKV2 now supports cross region
support.
* If endpoint is configured and it is not the central (s3.amazonaws.com)
one, parse the endpoint for the region. If unable to
parse region from endpoint (or endpoint is null), look at the the region
configuration.
* If region configured, set it in the client. If it is null, use US_EAST_1.
If config is empty string, use SDK region resolution chain.
### How was this patch tested?
test in progress with s3 london, -Dparallel-tests -DtestsThreadCount=10
expecting vpce test failure.
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Improve s3a region handling, including determining from endpoint
> ----------------------------------------------------------------
>
> Key: HADOOP-18908
> URL: https://issues.apache.org/jira/browse/HADOOP-18908
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
>
> s3a now requires the fs.s3a.endpoint.region to be set; and while it can
> determine it from a network call, this takes time and doesn't work for third
> party stores.
> proposed
> * reinstate parsing of the fs.3a.endpoint url to automatically determine
> region from well known endoints (and vplink ones)
> * don't try to talk to AWS S3 if endpoint isn't an aws one: for that caller
> must declare (HADOOP-18673)
> * document this in v2 migration, including stack traces of falures
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
