[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687623#comment-15687623
]
Hudson commented on HDFS-11148:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10873 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10873/])
HDFS-11148. Update DataNode to use StorageLocationChecker at startup. (arp: rev
613b902b9808c1c679674047cff15feade01dfab)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/checker/StorageLocationChecker.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/checker/TestStorageLocationChecker.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeUUID.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataDirs.java
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Fix For: 3.0.0-alpha2
>
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687524#comment-15687524
]
ASF GitHub Bot commented on HDFS-11148:
---
Github user anuengineer commented on the issue:
https://github.com/apache/hadoop/pull/162
Arpit, thanks for updating the patch. +1. LGTM.
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679962#comment-15679962
]
Hadoop QA commented on HDFS-11148:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 285 unchanged - 1 fixed = 286 total (was 286) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m
57s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HDFS-11148 |
| GITHUB PR | https://github.com/apache/hadoop/pull/162 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a49045892615 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7584fbf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/17616/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17616/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/17616/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15678348#comment-15678348
]
Hadoop QA commented on HDFS-11148:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 26s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 4 new + 285 unchanged - 1 fixed = 289 total (was 286) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}106m 2s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HDFS-11148 |
| GITHUB PR | https://github.com/apache/hadoop/pull/162 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux c8847131fb2d 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7584fbf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/17611/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/17611/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/17611/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17611/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/17611/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15678120#comment-15678120
]
Hadoop QA commented on HDFS-11148:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 27s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 3 new + 285 unchanged - 1 fixed = 288 total (was 286) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 73m 35s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 93m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.web.TestWebHDFS |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HDFS-11148 |
| GITHUB PR | https://github.com/apache/hadoop/pull/162 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux faaa6181bf71 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7584fbf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/17610/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/17610/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/17610/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17610/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/17610/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672478#comment-15672478
]
Hadoop QA commented on HDFS-11148:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 27s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 2 new + 299 unchanged - 1 fixed = 301 total (was 300) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 38s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 85m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.checker.TestStorageLocationChecker |
| | hadoop.hdfs.server.namenode.TestReconstructStripedBlocks |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HDFS-11148 |
| GITHUB PR | https://github.com/apache/hadoop/pull/162 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b805ed6f2ab2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 04a024b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/17596/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/17596/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/17596/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17596/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672322#comment-15672322
]
Arpit Agarwal commented on HDFS-11148:
--
Thank you for the feedback [~anu]. I have reduced the shutdown wait time and
fixed unit tests via an addendum commit.
https://github.com/arp7/hadoop/commit/f2819503ee7dcf6b996163cbc0edd12747d9ddc4
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672133#comment-15672133
]
Arpit Agarwal commented on HDFS-11148:
--
The test failures look related to the patch. I will update the pull request.
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672093#comment-15672093
]
ASF GitHub Bot commented on HDFS-11148:
---
Github user arp7 commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/162#discussion_r88357779
--- Diff:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataDirs.java
---
@@ -96,26 +91,4 @@ public void testDataDirParsing() throws Throwable {
assertThat(locations.get(1).getStorageType(), is(StorageType.DISK));
assertThat(locations.get(1).getUri(), is(dir1.toURI()));
}
-
- @Test(timeout = 3)
- public void testDataDirValidation() throws Throwable {
--- End diff --
Yes there's new tests in TestStorageLocationChecker added by HDFS-9
which cover this better with different combinations of failed/healthy storages.
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672084#comment-15672084
]
ASF GitHub Bot commented on HDFS-11148:
---
Github user arp7 commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/162#discussion_r88357293
--- Diff:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
---
@@ -1935,6 +1937,10 @@ public void shutdown() {
}
}
+if (storageLocationChecker != null) {
+ storageLocationChecker.shutdownAndWait(10, TimeUnit.SECONDS);
--- End diff --
Good point. We can exit much quicker since a stalled disk check need not
block shutdown at all. I'll change it to 1 second.
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672069#comment-15672069
]
Anu Engineer commented on HDFS-11148:
-
+1, LGTM.
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672040#comment-15672040
]
Hadoop QA commented on HDFS-11148:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 30s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 249 unchanged - 1 fixed = 250 total (was 250) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 7s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}107m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HDFS-11148 |
| GITHUB PR | https://github.com/apache/hadoop/pull/162 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 22dca024f313 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c90891e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/17592/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/17592/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/17592/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17592/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672026#comment-15672026
]
ASF GitHub Bot commented on HDFS-11148:
---
Github user anuengineer commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/162#discussion_r88354093
--- Diff:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataDirs.java
---
@@ -96,26 +91,4 @@ public void testDataDirParsing() throws Throwable {
assertThat(locations.get(1).getStorageType(), is(StorageType.DISK));
assertThat(locations.get(1).getUri(), is(dir1.toURI()));
}
-
- @Test(timeout = 3)
- public void testDataDirValidation() throws Throwable {
--- End diff --
Presuming that we will have an equivalent or better test using
storageLocationChecker ?
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672025#comment-15672025
]
ASF GitHub Bot commented on HDFS-11148:
---
Github user anuengineer commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/162#discussion_r88354002
--- Diff:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
---
@@ -1935,6 +1937,10 @@ public void shutdown() {
}
}
+if (storageLocationChecker != null) {
+ storageLocationChecker.shutdownAndWait(10, TimeUnit.SECONDS);
--- End diff --
Something for you to consider, most of the other threads seems to shutdown
much quicker, do we need to wait 10 seconds ?
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking Storage Locations at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11148) Update DataNode to use StorageLocationChecker at startup
[
https://issues.apache.org/jira/browse/HDFS-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671690#comment-15671690
]
ASF GitHub Bot commented on HDFS-11148:
---
GitHub user arp7 opened a pull request:
https://github.com/apache/hadoop/pull/162
HDFS-11148. Update DataNode to use StorageLocationChecker at startup.
Change-Id: I664e5c719921ef5e0891bc392f37ee67639a8660
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/arp7/hadoop HDFS-11148
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hadoop/pull/162.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #162
> Update DataNode to use StorageLocationChecker at startup
>
>
> Key: HDFS-11148
> URL: https://issues.apache.org/jira/browse/HDFS-11148
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
>
> The DataNode can use the {{StorageLocationChecker}} introduced by HDFS-9
> to parallelize checking {{StorageLocation}}s at process startup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
