[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory
[ https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907417#comment-16907417 ] Siddharth Seth commented on HADOOP-16505: - [~viczsaurav] - any thoughts on how this compares to https://issues.apache.org/jira/browse/HADOOP-16445 That sets up a new config to register "signerName:signerClass" pairs, instead of re-using the current config to allow class names. > Add ability to register custom signer with AWS SignerFactory > > > Key: HADOOP-16505 > URL: https://issues.apache.org/jira/browse/HADOOP-16505 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, hadoop-aws >Affects Versions: 3.3.0 >Reporter: Saurav Verma >Assignee: Saurav Verma >Priority: Major > Attachments: HADOOP-16505.patch, hadoop-16505-1.patch > > > Currently, the AWS SignerFactory restricts the class of Signer algorithms > that can be used. > We require an ability to register a custom Signer. The SignerFactory supports > this functionality through its {{registerSigner}} method. > By providing a fully qualified classname to the existing parameter > {{fs.s3a.signing-algorithm}}, the custom signer can be registered. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory
[ https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907379#comment-16907379 ] Saurav Verma commented on HADOOP-16505: --- Thanks [~jojochuang] and [~gabor.bota] for checking it. https://github.com/apache/hadoop/pull/1280 is indeed the PR for the same issue. I have written the unit test and was able to test it for eu-central-1 region. Please check in the PR. I am just running running {{dev-support/bin/test-patch}} locally once > Add ability to register custom signer with AWS SignerFactory > > > Key: HADOOP-16505 > URL: https://issues.apache.org/jira/browse/HADOOP-16505 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, hadoop-aws >Affects Versions: 3.3.0 >Reporter: Saurav Verma >Assignee: Saurav Verma >Priority: Major > Attachments: HADOOP-16505.patch, hadoop-16505-1.patch > > > Currently, the AWS SignerFactory restricts the class of Signer algorithms > that can be used. > We require an ability to register a custom Signer. The SignerFactory supports > this functionality through its {{registerSigner}} method. > By providing a fully qualified classname to the existing parameter > {{fs.s3a.signing-algorithm}}, the custom signer can be registered. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory
[ https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907281#comment-16907281 ] Gabor Bota commented on HADOOP-16505: - Thanks for working on this [~viczsaurav], and [~jojochuang] to notifying us! I think that the PR must be the current, because there is at least one test for this included in that. [~viczsaurav], could you include an integration test for this change? Also please run all integration test against an aws endpoint with these parameters at least: {{mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Ds3guard -Ddynamo}}, and tell us if it was successful. It would nice to show that the tests won't fail with the signer changed. > Add ability to register custom signer with AWS SignerFactory > > > Key: HADOOP-16505 > URL: https://issues.apache.org/jira/browse/HADOOP-16505 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, hadoop-aws >Affects Versions: 3.3.0 >Reporter: Saurav Verma >Assignee: Saurav Verma >Priority: Major > Attachments: HADOOP-16505.patch, hadoop-16505-1.patch > > > Currently, the AWS SignerFactory restricts the class of Signer algorithms > that can be used. > We require an ability to register a custom Signer. The SignerFactory supports > this functionality through its {{registerSigner}} method. > By providing a fully qualified classname to the existing parameter > {{fs.s3a.signing-algorithm}}, the custom signer can be registered. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory
[ https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907217#comment-16907217 ] Wei-Chiu Chuang commented on HADOOP-16505: -- [~gabor.bota] [~ste...@apache.org] would you like to review this one? Note there's a PR https://github.com/apache/hadoop/pull/1280 open for the same. I'm not sure which one is current. > Add ability to register custom signer with AWS SignerFactory > > > Key: HADOOP-16505 > URL: https://issues.apache.org/jira/browse/HADOOP-16505 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, hadoop-aws >Affects Versions: 3.3.0 >Reporter: Saurav Verma >Assignee: Saurav Verma >Priority: Major > Attachments: HADOOP-16505.patch, hadoop-16505-1.patch > > > Currently, the AWS SignerFactory restricts the class of Signer algorithms > that can be used. > We require an ability to register a custom Signer. The SignerFactory supports > this functionality through its {{registerSigner}} method. > By providing a fully qualified classname to the existing parameter > {{fs.s3a.signing-algorithm}}, the custom signer can be registered. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory
[ https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907175#comment-16907175 ] Hadoop QA commented on HADOOP-16505: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 36s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 33s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 17s{color} | {color:orange} hadoop-tools/hadoop-aws: The patch generated 17 new + 7 unchanged - 1 fixed = 24 total (was 8) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 5s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 17s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 34s{color} | {color:red} The patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 54m 27s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 | | JIRA Issue | HADOOP-16505 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12977573/HADOOP-16505.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 47c118e0a439 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 0e4b757 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_212 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/16480/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/16480/testReport/ | | asflicense | https://builds.apache.org/job/PreCommit-HADOOP-Build/16480/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 308 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output |
[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory
[ https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905247#comment-16905247 ] Hadoop QA commented on HADOOP-16505: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 54s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 9s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 20s{color} | {color:orange} hadoop-tools/hadoop-aws: The patch generated 2 new + 6 unchanged - 1 fixed = 8 total (was 7) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 58s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 47s{color} | {color:red} hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 7s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 61m 46s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-tools/hadoop-aws | | | Dead store to e in org.apache.hadoop.fs.s3a.S3AUtils.initConnectionSettings(Configuration, ClientConfiguration) At S3AUtils.java:org.apache.hadoop.fs.s3a.S3AUtils.initConnectionSettings(Configuration, ClientConfiguration) At S3AUtils.java:[line 1245] | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e53b4 | | JIRA Issue | HADOOP-16505 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12977357/hadoop-16505-1.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 561c5f493664 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / e4b538b | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_212 | | findbugs | v3.1.0-RC1 | |
[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory
[ https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905204#comment-16905204 ] Saurav Verma commented on HADOOP-16505: --- Contributor : Brian Kelly, Saurav Verma > Add ability to register custom signer with AWS SignerFactory > > > Key: HADOOP-16505 > URL: https://issues.apache.org/jira/browse/HADOOP-16505 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, hadoop-aws >Affects Versions: 3.3.0 >Reporter: Saurav Verma >Priority: Major > Attachments: hadoop-16505-1.patch > > > Currently, the AWS SignerFactory restricts the class of Signer algorithms > that can be used. > We require an ability to register a custom Signer. The SignerFactory supports > this functionality through its {{registerSigner}} method. > By providing a fully qualified classname to the existing parameter > {{fs.s3a.signing-algorithm}}, the custom signer can be registered. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org