[jira] [Commented] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider in trunk; deprecate in branch-2
[
https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964957#comment-15964957
]
Mingliang Liu commented on HADOOP-14248:
Thanks [~cnauroth] for taking care of this. I'm totally +1 on the proposal.
> Retire SharedInstanceProfileCredentialsProvider in trunk; deprecate in
> branch-2
> ---
>
> Key: HADOOP-14248
> URL: https://issues.apache.org/jira/browse/HADOOP-14248
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.0.0-alpha3
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HADOOP-14248.000.patch, HADOOP-14248.001.patch,
> HADOOP-14248-branch-2.001.patch, HADOOP-14248-branch-2.002.patch
>
>
> This is from the discussion in [HADOOP-13050].
> So [HADOOP-13727] added the SharedInstanceProfileCredentialsProvider, which
> effectively reduces high number of connections to EC2 Instance Metadata
> Service caused by InstanceProfileCredentialsProvider. That patch, in order to
> prevent the throttling problem, defined new class
> {{SharedInstanceProfileCredentialsProvider}} as a subclass of
> {{InstanceProfileCredentialsProvider}}, which enforces creation of only a
> single instance.
> Per [HADOOP-13050], we upgraded the AWS Java SDK. Since then, the
> {{InstanceProfileCredentialsProvider}} in SDK code internally enforces a
> singleton. That confirms that our effort in [HADOOP-13727] makes 100% sense.
> Meanwhile, {{SharedInstanceProfileCredentialsProvider}} can retire gracefully
> in trunk branch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider in trunk; deprecate in branch-2
[
https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964937#comment-15964937
]
Chris Nauroth commented on HADOOP-14248:
[~liuml07], this looks good now, and I confirmed a full test run against
US-west-2.
However, I'm now thinking that we need two separates JIRA issues, just for the
sake of accurate tracking against target versions and release notes.
HADOOP-14248 would track the removal from trunk (already covered by the current
release note), and the new issue would be targeted to 2.9.0 with a different
release note describing deprecation instead of removal. (No need to repeat
pre-commit. I'd just comment on the new JIRA that pre-commit for branch-2 was
already covered here.)
Do you think that makes sense? If so, I'd be happy to be the JIRA janitor and
finish off committing this. :-)
> Retire SharedInstanceProfileCredentialsProvider in trunk; deprecate in
> branch-2
> ---
>
> Key: HADOOP-14248
> URL: https://issues.apache.org/jira/browse/HADOOP-14248
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.0.0-alpha3
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HADOOP-14248.000.patch, HADOOP-14248.001.patch,
> HADOOP-14248-branch-2.001.patch, HADOOP-14248-branch-2.002.patch
>
>
> This is from the discussion in [HADOOP-13050].
> So [HADOOP-13727] added the SharedInstanceProfileCredentialsProvider, which
> effectively reduces high number of connections to EC2 Instance Metadata
> Service caused by InstanceProfileCredentialsProvider. That patch, in order to
> prevent the throttling problem, defined new class
> {{SharedInstanceProfileCredentialsProvider}} as a subclass of
> {{InstanceProfileCredentialsProvider}}, which enforces creation of only a
> single instance.
> Per [HADOOP-13050], we upgraded the AWS Java SDK. Since then, the
> {{InstanceProfileCredentialsProvider}} in SDK code internally enforces a
> singleton. That confirms that our effort in [HADOOP-13727] makes 100% sense.
> Meanwhile, {{SharedInstanceProfileCredentialsProvider}} can retire gracefully
> in trunk branch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider in trunk; deprecate in branch-2
[
https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964891#comment-15964891
]
Hadoop QA commented on HADOOP-14248:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
48s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
5s{color} | {color:green} branch-2 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
50s{color} | {color:green} branch-2 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
28s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} branch-2 passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} branch-2 passed with JDK v1.7.0_121 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
39s{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 5m 39s{color}
| {color:red} root-jdk1.8.0_121 with JDK v1.8.0_121 generated 2 new + 899
unchanged - 1 fixed = 901 total (was 900) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
39s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 6m 39s{color}
| {color:red} root-jdk1.7.0_121 with JDK v1.7.0_121 generated 2 new + 992
unchanged - 1 fixed = 994 total (was 993) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed with JDK v1.8.0_121 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
5s{color} | {color:green} hadoop-common in the patch passed with JDK
v1.7.0_121. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_121.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
[jira] [Commented] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider in trunk; deprecate in branch-2
[
https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963537#comment-15963537
]
Chris Nauroth commented on HADOOP-14248:
Hello [~liuml07]. This looks good overall. I have a comment on the branch-2
patch.
{code}
private SharedInstanceProfileCredentialsProvider() {
-super();
+InstanceProfileCredentialsProvider.getInstance();
}
{code}
I don't think this change is necessary. The call to
{{InstanceProfileCredentialsProvider#getInstance()}} returns an instance
(always the same one now that we've upgraded the AWS SDK), but then it never
saves a reference to that instance or does anything else with it.
> Retire SharedInstanceProfileCredentialsProvider in trunk; deprecate in
> branch-2
> ---
>
> Key: HADOOP-14248
> URL: https://issues.apache.org/jira/browse/HADOOP-14248
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.0.0-alpha3
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HADOOP-14248.000.patch, HADOOP-14248.001.patch,
> HADOOP-14248-branch-2.001.patch
>
>
> This is from the discussion in [HADOOP-13050].
> So [HADOOP-13727] added the SharedInstanceProfileCredentialsProvider, which
> effectively reduces high number of connections to EC2 Instance Metadata
> Service caused by InstanceProfileCredentialsProvider. That patch, in order to
> prevent the throttling problem, defined new class
> {{SharedInstanceProfileCredentialsProvider}} as a subclass of
> {{InstanceProfileCredentialsProvider}}, which enforces creation of only a
> single instance.
> Per [HADOOP-13050], we upgraded the AWS Java SDK. Since then, the
> {{InstanceProfileCredentialsProvider}} in SDK code internally enforces a
> singleton. That confirms that our effort in [HADOOP-13727] makes 100% sense.
> Meanwhile, {{SharedInstanceProfileCredentialsProvider}} can retire gracefully
> in trunk branch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider in trunk; Deprecate in branch-2
[
https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946119#comment-15946119
]
Hadoop QA commented on HADOOP-14248:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
12s{color} | {color:green} root generated 0 new + 776 unchanged - 1 fixed = 776
total (was 777) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 37s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}101m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-14248 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12860939/HADOOP-14248.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux cc86f2b4f7c0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 01aca54 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11955/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
