[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938751#comment-15938751 ] Hudson commented on HADOOP-14213: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11446 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11446/]) HADOOP-14213. Move Configuration runtime check for hadoop-site.xml to (raviprak: rev 595f62e362c08704d6fb692e21c97b512bc7ec49) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.8.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: HADOOP-14213.1.patch, HADOOP-14213.2.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938700#comment-15938700 ] Ravi Prakash commented on HADOOP-14213: --- Thanks for the clarification Jon! LGTM. +1. Since you have not removed hadoop-site from default resources, my understanding is that this is *not* a backward incompatible change. Will commit shortly to trunk and branch-2. > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: HADOOP-14213.1.patch, HADOOP-14213.2.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938572#comment-15938572 ] Jonathan Eagles commented on HADOOP-14213: -- Both TestKDiag and TestDNS are intermittent failures not related to this patch. The overall improvement to Configuration loading on my box is around 2.5% which is very tiny by itself, but a few changes like this will make a big difference. What else is needed to get this patch in? > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: HADOOP-14213.1.patch, HADOOP-14213.2.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[
https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937851#comment-15937851
]
Hadoop QA commented on HADOOP-14213:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 1s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 22s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
| | hadoop.net.TestDNS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-14213 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12860079/HADOOP-14213.2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 8cd44d8121f8 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 59d6925 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11892/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11892/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11892/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Move Configuration runtime check for hadoop-site.xml to initialization
> --
>
> Key: HADOOP-14213
> URL: https://issues.apache.org/jira/browse/HADOOP-14213
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937798#comment-15937798 ] Jonathan Eagles commented on HADOOP-14213: -- To elaborate on the precedence, currently the final parameters of core-default.xml and core-site.xml take precedence over hadoop-site.xml and this patch preserves that ordering. It is correct that the hadoop-site.xml configs will override for non-final parameters. That behavior is preserved as well. > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: HADOOP-14213.1.patch, HADOOP-14213.2.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937750#comment-15937750 ] Ravi Prakash commented on HADOOP-14213: --- Thanks Jeagles! # typo : "hadoop-site.xml.xml" -> "hadoop-site.xml" ?? # I'm sorry! Could you please elaborate how core-site would take precedence over hadoop-site still? It seems like hadoop-site would override the attributes in core-site, right? > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: HADOOP-14213.1.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937091#comment-15937091 ] Jonathan Eagles commented on HADOOP-14213: -- The test failure in TestZKFailoverController was intermittent and not related to this patch. > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: HADOOP-14213.1.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[
https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936979#comment-15936979
]
Hadoop QA commented on HADOOP-14213:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 23s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-14213 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12859971/HADOOP-14213.1.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 111c6b342c77 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f462e1f |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11880/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11880/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11880/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Move Configuration runtime check for hadoop-site.xml to initialization
> --
>
> Key: HADOOP-14213
> URL: https://issues.apache.org/jira/browse/HADOOP-14213
>
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936921#comment-15936921 ] Steve Loughran commented on HADOOP-14213: - Static init makes sense, currently there's one per instance, which adds up > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: HADOOP-14213.1.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936790#comment-15936790 ] Jonathan Eagles commented on HADOOP-14213: -- This patch moves the decision of using hadoop-site.xml as a default resource to the static initialization block in Configuration. This patch maintains the previous precedence ordering of the core-default.xml, core-site.xml, and hadoop-site.xml. > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Attachments: HADOOP-14213.1.patch > > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14213) Move Configuration runtime check for hadoop-site.xml to initialization
[ https://issues.apache.org/jira/browse/HADOOP-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936745#comment-15936745 ] Allen Wittenauer commented on HADOOP-14213: --- bq. It has been long deprecated and is not present in most if not nearly all installations. We just need to be cognizant of the fact that... bq. remove hadoop-site.xml as a default resource ... is an incompatible change which couldn't go into branch-2. (FWIW, I'll raise my hand as one of the folks who still uses hadoop-site.xml on occasion for non-prod installs since it's easier than dealing with the complete mess that happened after the project split.) > Move Configuration runtime check for hadoop-site.xml to initialization > -- > > Key: HADOOP-14213 > URL: https://issues.apache.org/jira/browse/HADOOP-14213 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > > Each Configuration object that loads defaults checks for hadoop-site.xml. It > has been long deprecated and is not present in most if not nearly all > installations. The getResource check for hadoop-site.xml has to check the > entire classpath since it is not found. This jira proposes to 1) either > remove hadoop-site.xml as a default resource or 2) move the check to static > initialization of the class so the performance hit is only taken once. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
