[
https://issues.apache.org/jira/browse/HADOOP-19286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884325#comment-17884325
]
ASF GitHub Bot commented on HADOOP-19286:
-----------------------------------------
steveloughran commented on code in PR #7067:
URL: https://github.com/apache/hadoop/pull/7067#discussion_r1773578299
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEndpointRegion.java:
##########
@@ -346,6 +347,37 @@ public void
testCentralEndpointAndDifferentRegionThanBucket() throws Throwable {
assertRequesterPaysFileExistence(newConf);
}
+ @Test
+ public void testWithOutCrossRegionAccess() throws Exception {
+ describe("Verify cross region access fails when disabled");
+ final Configuration newConf = new Configuration(getConfiguration());
+ // skip the test if the region is eu-west-2
+ String region =
getFileSystem().getS3AInternals().getBucketMetadata().bucketRegion();
+ if (EU_WEST_2.equals(region)) {
+ return;
+ }
+ // disable cross region access
+ newConf.setBoolean(AWS_S3_CROSS_REGION_ACCESS_ENABLED, false);
+ newConf.set(AWS_REGION, EU_WEST_2);
+ S3AFileSystem fs = new S3AFileSystem();
+ fs.initialize(getFileSystem().getUri(), newConf);
+ intercept(AWSRedirectException.class,
+ "does not match the AWS region containing the bucket",
+ () -> fs.exists(getFileSystem().getWorkingDirectory()));
+ }
+
+ @Test
+ public void testWithCrossRegionAccess() throws Exception {
+ describe("Verify cross region access succeed when enabled");
+ final Configuration newConf = new Configuration(getConfiguration());
+ // enable cross region access
+ newConf.setBoolean(AWS_S3_CROSS_REGION_ACCESS_ENABLED, true);
+ newConf.set(AWS_REGION, EU_WEST_2);
Review Comment:
same comments as above
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEndpointRegion.java:
##########
@@ -346,6 +347,37 @@ public void
testCentralEndpointAndDifferentRegionThanBucket() throws Throwable {
assertRequesterPaysFileExistence(newConf);
}
+ @Test
+ public void testWithOutCrossRegionAccess() throws Exception {
+ describe("Verify cross region access fails when disabled");
+ final Configuration newConf = new Configuration(getConfiguration());
+ // skip the test if the region is eu-west-2
+ String region =
getFileSystem().getS3AInternals().getBucketMetadata().bucketRegion();
+ if (EU_WEST_2.equals(region)) {
Review Comment:
1. I'd like a different region here a that is my region and I would like the
test coverage.
2. what happens with third party stores?
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEndpointRegion.java:
##########
@@ -346,6 +347,37 @@ public void
testCentralEndpointAndDifferentRegionThanBucket() throws Throwable {
assertRequesterPaysFileExistence(newConf);
}
+ @Test
+ public void testWithOutCrossRegionAccess() throws Exception {
+ describe("Verify cross region access fails when disabled");
+ final Configuration newConf = new Configuration(getConfiguration());
+ // skip the test if the region is eu-west-2
+ String region =
getFileSystem().getS3AInternals().getBucketMetadata().bucketRegion();
+ if (EU_WEST_2.equals(region)) {
+ return;
+ }
+ // disable cross region access
+ newConf.setBoolean(AWS_S3_CROSS_REGION_ACCESS_ENABLED, false);
+ newConf.set(AWS_REGION, EU_WEST_2);
+ S3AFileSystem fs = new S3AFileSystem();
+ fs.initialize(getFileSystem().getUri(), newConf);
+ intercept(AWSRedirectException.class,
+ "does not match the AWS region containing the bucket",
+ () -> fs.exists(getFileSystem().getWorkingDirectory()));
+ }
+
+ @Test
+ public void testWithCrossRegionAccess() throws Exception {
+ describe("Verify cross region access succeed when enabled");
+ final Configuration newConf = new Configuration(getConfiguration());
+ // enable cross region access
+ newConf.setBoolean(AWS_S3_CROSS_REGION_ACCESS_ENABLED, true);
+ newConf.set(AWS_REGION, EU_WEST_2);
+ S3AFileSystem fs = new S3AFileSystem();
Review Comment:
needs to be in try-with-resources to close()
> Support S3A cross region access when S3 region/endpoint is set
> --------------------------------------------------------------
>
> Key: HADOOP-19286
> URL: https://issues.apache.org/jira/browse/HADOOP-19286
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Reporter: Syed Shameerur Rahman
> Assignee: Syed Shameerur Rahman
> Priority: Major
> Labels: pull-request-available
>
> Currently when S3 region nor endpoint is set, the default region is set to
> us-east-2 with cross region access enabled. But when region or endpoint is
> set, cross region access is not enabled.
> The proposal here is to carves out cross region access as a separate config
> and enable/disable it irrespective of region/endpoint is set. This gives more
> flexibility to the user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
