[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory

2019-08-14 Thread Siddharth Seth (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907417#comment-16907417
 ] 

Siddharth Seth commented on HADOOP-16505:
-

[~viczsaurav] - any thoughts on how this compares to 
https://issues.apache.org/jira/browse/HADOOP-16445

That sets up a new config to register "signerName:signerClass" pairs, instead 
of re-using the current config to allow class names.

> Add ability to register custom signer with AWS SignerFactory
> 
>
> Key: HADOOP-16505
> URL: https://issues.apache.org/jira/browse/HADOOP-16505
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs/s3, hadoop-aws
>Affects Versions: 3.3.0
>Reporter: Saurav Verma
>Assignee: Saurav Verma
>Priority: Major
> Attachments: HADOOP-16505.patch, hadoop-16505-1.patch
>
>
> Currently, the AWS SignerFactory restricts the class of Signer algorithms 
> that can be used. 
> We require an ability to register a custom Signer. The SignerFactory supports 
> this functionality through its {{registerSigner}} method. 
> By providing a fully qualified classname to the existing parameter 
> {{fs.s3a.signing-algorithm}}, the custom signer can be registered.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory

2019-08-14 Thread Saurav Verma (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907379#comment-16907379
 ] 

Saurav Verma commented on HADOOP-16505:
---

Thanks [~jojochuang] and [~gabor.bota] for checking it.
https://github.com/apache/hadoop/pull/1280 is indeed the PR for the same issue. 
I have written the unit test and was able to test it for eu-central-1 region. 
Please check in the PR.
I am just running  running {{dev-support/bin/test-patch}} locally once

> Add ability to register custom signer with AWS SignerFactory
> 
>
> Key: HADOOP-16505
> URL: https://issues.apache.org/jira/browse/HADOOP-16505
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs/s3, hadoop-aws
>Affects Versions: 3.3.0
>Reporter: Saurav Verma
>Assignee: Saurav Verma
>Priority: Major
> Attachments: HADOOP-16505.patch, hadoop-16505-1.patch
>
>
> Currently, the AWS SignerFactory restricts the class of Signer algorithms 
> that can be used. 
> We require an ability to register a custom Signer. The SignerFactory supports 
> this functionality through its {{registerSigner}} method. 
> By providing a fully qualified classname to the existing parameter 
> {{fs.s3a.signing-algorithm}}, the custom signer can be registered.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory

2019-08-14 Thread Gabor Bota (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907281#comment-16907281
 ] 

Gabor Bota commented on HADOOP-16505:
-

Thanks for working on this [~viczsaurav], and [~jojochuang] to notifying us!

I think that the PR must be the current, because there is at least one test for 
this included in that.

[~viczsaurav], could you include an integration test for this change? Also 
please run all integration test against an aws endpoint with these parameters 
at least: {{mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Ds3guard 
-Ddynamo}}, and tell us if it was successful. It would nice to show that the 
tests won't fail with the signer changed.


> Add ability to register custom signer with AWS SignerFactory
> 
>
> Key: HADOOP-16505
> URL: https://issues.apache.org/jira/browse/HADOOP-16505
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs/s3, hadoop-aws
>Affects Versions: 3.3.0
>Reporter: Saurav Verma
>Assignee: Saurav Verma
>Priority: Major
> Attachments: HADOOP-16505.patch, hadoop-16505-1.patch
>
>
> Currently, the AWS SignerFactory restricts the class of Signer algorithms 
> that can be used. 
> We require an ability to register a custom Signer. The SignerFactory supports 
> this functionality through its {{registerSigner}} method. 
> By providing a fully qualified classname to the existing parameter 
> {{fs.s3a.signing-algorithm}}, the custom signer can be registered.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory

2019-08-14 Thread Wei-Chiu Chuang (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907217#comment-16907217
 ] 

Wei-Chiu Chuang commented on HADOOP-16505:
--

[~gabor.bota] [~ste...@apache.org] would you like to review this one?
Note there's a PR https://github.com/apache/hadoop/pull/1280 open for the same. 
I'm not sure which one is current.

> Add ability to register custom signer with AWS SignerFactory
> 
>
> Key: HADOOP-16505
> URL: https://issues.apache.org/jira/browse/HADOOP-16505
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs/s3, hadoop-aws
>Affects Versions: 3.3.0
>Reporter: Saurav Verma
>Assignee: Saurav Verma
>Priority: Major
> Attachments: HADOOP-16505.patch, hadoop-16505-1.patch
>
>
> Currently, the AWS SignerFactory restricts the class of Signer algorithms 
> that can be used. 
> We require an ability to register a custom Signer. The SignerFactory supports 
> this functionality through its {{registerSigner}} method. 
> By providing a fully qualified classname to the existing parameter 
> {{fs.s3a.signing-algorithm}}, the custom signer can be registered.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory

2019-08-14 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907175#comment-16907175
 ] 

Hadoop QA commented on HADOOP-16505:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m 33s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 17s{color} | {color:orange} hadoop-tools/hadoop-aws: The patch generated 17 
new + 7 unchanged - 1 fixed = 24 total (was 8) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m  5s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m 
17s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
34s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 27s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16505 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12977573/HADOOP-16505.patch |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 47c118e0a439 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 
08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0e4b757 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16480/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16480/testReport/ |
| asflicense | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16480/artifact/out/patch-asflicense-problems.txt
 |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | 

[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory

2019-08-12 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905247#comment-16905247
 ] 

Hadoop QA commented on HADOOP-16505:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
54s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m 
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m  9s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
36s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 20s{color} | {color:orange} hadoop-tools/hadoop-aws: The patch generated 2 
new + 6 unchanged - 1 fixed = 8 total (was 7) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m 58s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
47s{color} | {color:red} hadoop-tools/hadoop-aws generated 1 new + 0 unchanged 
- 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m  
7s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
31s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 46s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-tools/hadoop-aws |
|  |  Dead store to e in 
org.apache.hadoop.fs.s3a.S3AUtils.initConnectionSettings(Configuration, 
ClientConfiguration)  At 
S3AUtils.java:org.apache.hadoop.fs.s3a.S3AUtils.initConnectionSettings(Configuration,
 ClientConfiguration)  At S3AUtils.java:[line 1245] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16505 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12977357/hadoop-16505-1.patch |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 561c5f493664 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e4b538b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| 

[jira] [Commented] (HADOOP-16505) Add ability to register custom signer with AWS SignerFactory

2019-08-12 Thread Saurav Verma (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905204#comment-16905204
 ] 

Saurav Verma commented on HADOOP-16505:
---

Contributor :  Brian Kelly, Saurav Verma

> Add ability to register custom signer with AWS SignerFactory
> 
>
> Key: HADOOP-16505
> URL: https://issues.apache.org/jira/browse/HADOOP-16505
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs/s3, hadoop-aws
>Affects Versions: 3.3.0
>Reporter: Saurav Verma
>Priority: Major
> Attachments: hadoop-16505-1.patch
>
>
> Currently, the AWS SignerFactory restricts the class of Signer algorithms 
> that can be used. 
> We require an ability to register a custom Signer. The SignerFactory supports 
> this functionality through its {{registerSigner}} method. 
> By providing a fully qualified classname to the existing parameter 
> {{fs.s3a.signing-algorithm}}, the custom signer can be registered.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org