steveloughran commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1512593751
##
hadoop-project/pom.xml:
##
@@ -1976,6 +1976,30 @@
log4j-web
${log4j2.version}
+
+software.amazon.s3.accessgrants
+
steveloughran commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1936006860
@adnanhemani thanks; without that change we'd have problems with the PR, as
in "you get to support it all through reflection" the way we have to do with
wildfly/openssl binding (Netw
adnanhemani commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1935463310
This has been evolved into #6544. This PR will no longer be used now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
adnanhemani commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1933245232
Update: I've successfully influenced the S3 Access Grants team and the AWS
Java SDK team to include the S3 Access Grants Plugin within the AWS Java SDK
bundle. For that, we will requir
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1928684496
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reex
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1928393471
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reex
jxhan3 commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1478681708
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -401,4 +409,32 @@ private static Region getS3RegionFromEndpoint(final
ahmarsuhail commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1478502544
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -401,4 +409,32 @@ private static Region getS3RegionFromEndpoint(
adnanhemani commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1926257843
Hi @steveloughran and @ahmarsuhail - I think this code is in a much more
ready state than before and we've attempted to answer the questions you had
earlier. Please let us know what ot
adnanhemani commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1477676885
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -401,4 +409,32 @@ private static Region getS3RegionFromEndpoint(
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1925476864
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexe
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1925460913
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexe
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1925326604
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexe
jxhan3 commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1925038842
Test optional plugin:
1. if plugin is in class path, same result
2. if plugin is not in class path:
```
hadoop@ip-172-31-35-222 ~]$ hadoop fs -ls
s3a://s3ag-pentests-us-e
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1924323299
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexe
adnanhemani commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1922468809
@jxhan3 can you please put the output from your console testing onto the
comments here? No need to paste the actual grants but just say which ones have
grants and should be successful
jxhan3 commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1922460753
Local verification is running, update soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t
jxhan3 commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1922460413
Some tests done in EMR cluster: (all successful with expected behavior)
1. set up AWS IAM roles with S3 Access Grants: read prefix, write prefix,
read/write prefix
2. test on us-east-1
jxhan3 commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1475274368
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java:
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Founda
jxhan3 commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1475271643
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -370,4 +375,18 @@ private static Region getS3RegionFromEndpoint(Strin
jxhan3 commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1475168380
##
hadoop-project/pom.xml:
##
@@ -187,7 +187,7 @@
1.0-beta-1
900
1.12.599
-2.23.5
+2.23.7
Review Comment:
removed in next version
--
This i
jxhan3 commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1475155889
##
hadoop-project/pom.xml:
##
@@ -187,7 +187,7 @@
1.0-beta-1
900
1.12.599
-2.23.5
Review Comment:
revert this as well
--
This is an automated m
jxhan3 commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1475155705
##
LICENSE-binary:
##
@@ -363,7 +363,7 @@ org.objenesis:objenesis:2.6
org.xerial.snappy:snappy-java:1.1.10.4
org.yaml:snakeyaml:2.0
org.wildfly.openssl:wildfly-openssl
adnanhemani commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1473617958
##
hadoop-tools/hadoop-aws/pom.xml:
##
@@ -508,6 +508,29 @@
bundle
compile
+
+ software.amazon.s3.accessgrants
Review Comment:
This
adnanhemani commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1920244731
Hi @steveloughran - thanks for your time on this. We appreciate it a lot.
I've responded to your comments inline below:
> We cannot have any more unshaded aws sdk jars as require
adnanhemani commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1473617958
##
hadoop-tools/hadoop-aws/pom.xml:
##
@@ -508,6 +508,29 @@
bundle
compile
+
+ software.amazon.s3.accessgrants
Review Comment:
This
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1920154192
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexe
hadoop-yetus commented on PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1919823492
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexe
steveloughran commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1473066304
##
hadoop-tools/hadoop-aws/pom.xml:
##
@@ -508,6 +508,29 @@
bundle
compile
+
Review Comment:
the plugin should go in bundle.jar. it's m
steveloughran commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1472896913
##
hadoop-tools/hadoop-aws/pom.xml:
##
@@ -508,6 +508,29 @@
bundle
compile
+
+ software.amazon.s3.accessgrants
Review Comment:
why
ahmarsuhail commented on code in PR #6507:
URL: https://github.com/apache/hadoop/pull/6507#discussion_r1472858212
##
hadoop-project/pom.xml:
##
@@ -187,7 +187,7 @@
1.0-beta-1
900
1.12.599
-2.23.5
Review Comment:
cut, SDK upgrade needs to happen separately
31 matches
Mail list logo
