[
https://issues.apache.org/jira/browse/HADOOP-19286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884654#comment-17884654
]
ASF GitHub Bot commented on HADOOP-19286:
-----------------------------------------
steveloughran commented on code in PR #7067:
URL: https://github.com/apache/hadoop/pull/7067#discussion_r1775498471
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEndpointRegion.java:
##########
@@ -510,6 +514,16 @@ public void testCentralEndpointAndNullRegionFipsWithCRUD()
throws Throwable {
assertOpsUsingNewFs();
}
+ /**
+ * Skip the test if the region is sa-east-1.
+ */
+ private void skipCrossRegionTest() throws IOException {
+ String region =
getFileSystem().getS3AInternals().getBucketMetadata().bucketRegion();
+ if (SA_EAST_1.equals(region)) {
Review Comment:
needs a story for testing with third party stores. There I have a region
like "unknown" or "test" (but not empty string)
Maybe, rather than be clever here, declare that for testing the region
should be declared as "non-aws", with
* the new string stuck in S3ATestConstants
* the skip test here expanded to check for it
* the testing.md doc updated to cover this
everything else will take any string as a region, at least of those I've
tested
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEndpointRegion.java:
##########
@@ -350,32 +353,33 @@ public void
testCentralEndpointAndDifferentRegionThanBucket() throws Throwable {
@Test
public void testWithOutCrossRegionAccess() throws Exception {
describe("Verify cross region access fails when disabled");
+ // skip the test if the region is sa-east-1
+ skipCrossRegionTest();
final Configuration newConf = new Configuration(getConfiguration());
- // skip the test if the region is eu-west-2
- String region =
getFileSystem().getS3AInternals().getBucketMetadata().bucketRegion();
- if (EU_WEST_2.equals(region)) {
- return;
- }
// disable cross region access
newConf.setBoolean(AWS_S3_CROSS_REGION_ACCESS_ENABLED, false);
- newConf.set(AWS_REGION, EU_WEST_2);
- S3AFileSystem fs = new S3AFileSystem();
- fs.initialize(getFileSystem().getUri(), newConf);
- intercept(AWSRedirectException.class,
- "does not match the AWS region containing the bucket",
- () -> fs.exists(getFileSystem().getWorkingDirectory()));
+ newConf.set(AWS_REGION, SA_EAST_1);
+ try (S3AFileSystem fs = new S3AFileSystem()) {
+ fs.initialize(getFileSystem().getUri(), newConf);
+ intercept(AWSRedirectException.class,
+ "does not match the AWS region containing the bucket",
+ () -> fs.exists(getFileSystem().getWorkingDirectory()));
+ }
}
@Test
public void testWithCrossRegionAccess() throws Exception {
describe("Verify cross region access succeed when enabled");
+ // skip the test if the region is sa-east-1
+ skipCrossRegionTest();
final Configuration newConf = new Configuration(getConfiguration());
Review Comment:
call
```
removeBaseAndBucketOverrides(newConf,
AWS_S3_CROSS_REGION_ACCESS_ENABLE,
AWS_REGION)
```
needed to strip out per bucket settings of these options.
> Support S3A cross region access when S3 region/endpoint is set
> --------------------------------------------------------------
>
> Key: HADOOP-19286
> URL: https://issues.apache.org/jira/browse/HADOOP-19286
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Syed Shameerur Rahman
> Assignee: Syed Shameerur Rahman
> Priority: Major
> Labels: pull-request-available
>
> Currently when S3 region nor endpoint is set, the default region is set to
> us-east-2 with cross region access enabled. But when region or endpoint is
> set, cross region access is not enabled.
> The proposal here is to carves out cross region access as a separate config
> and enable/disable it irrespective of region/endpoint is set. This gives more
> flexibility to the user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
