[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806019#comment-15806019
]
Hadoop QA commented on HDFS-8694:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HDFS-8694 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-8694 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/18085/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Expose the stats of IOErrors on each FsVolume through JMX
> -
>
> Key: HDFS-8694
> URL: https://issues.apache.org/jira/browse/HDFS-8694
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 2.7.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
>
>
> Currently, once DataNode hits an {{IOError}} when writing / reading block
> files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
> thread successfully finishes, DN does not record this {{IOError}}.
> We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025274#comment-15025274
]
Hadoop QA commented on HDFS-8694:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HDFS-8694 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch |
| JIRA Issue | HDFS-8694 |
| Powered by | Apache Yetus http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/13642/console |
This message was automatically generated.
> Expose the stats of IOErrors on each FsVolume through JMX
> -
>
> Key: HDFS-8694
> URL: https://issues.apache.org/jira/browse/HDFS-8694
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 2.7.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
>
>
> Currently, once DataNode hits an {{IOError}} when writing / reading block
> files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
> thread successfully finishes, DN does not record this {{IOError}}.
> We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628371#comment-14628371
]
Lei (Eddy) Xu commented on HDFS-8694:
-
Thanks for the reviews, [~andrew.wang]
bq. I have a hard time understanding when we should call handle the disk error
vs. just bubbling up, since it bubbles there seems like a danger of handling
the same root IOE more than once. What's the methodology here? Is it possible
to move handling to the top-level somewhere? I can manually examine all the
current callsites and callers, but that's not very future-proof.
The reason that call {{volume#handleIOErrors()}} is that when the {{IOE}} pops
up to the place we used to call {{DataNode#checkDiskErrorAsync()}}, the context
(IOs on which volume) is usually missing. My intention was to call
{{volume#handleIOErrors()}} at the highest level that manages {{volume}} object
lifetime. I will try to get rid of {{DataNode#checkDiskErrorAsync()}} call in a
following JIRA.
bq. Since we now have the volume as context, we should really move the disk
checker to be per-volume rather than DN wide. One volume throwing an error is
no reason to check all of them. This can be deferred to a follow-up; I think
it's a slam dunk.
Yes. It is the reason to put {{hadnleIOErrors()}} in to {{FsVolumeSpi}}. I was
thinking to use a per-volume thread to do {{checkDirs()}} and also use
{{numOfErrors()}} as trigger. I will do it in a following JIRA as well.
Working on the rest of comments.
Thanks a lot for these great comments.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14626770#comment-14626770
]
Andrew Wang commented on HDFS-8694:
---
Hi Eddy, overall this patch looks great, thanks for working on it. Some review
comments:
High-level:
* I have a hard time understanding when we should call handle the disk error
vs. just bubbling up, since it bubbles there seems like a danger of handling
the same root IOE more than once. What's the methodology here? Is it possible
to move handling to the top-level somewhere? I can manually examine all the
current callsites and callers, but that's not very future-proof.
* Related to the above, our unit tests do not cover anywhere close to all the
locations that handle an IOError. Adding tests for all of these would be
onerous, so very interested in a solution to the above.
* Since we now have the volume as context, we should really move the disk
checker to be per-volume rather than DN wide. One volume throwing an error is
no reason to check all of them. This can be deferred to a follow-up; I think
it's a slam dunk.
Nits:
* FsDatasetImpl#moveBlockAcrossStorage, can we get rid of volume and move
targetVolume's declaration outside of the try? Looks equal to volume.
* In places like BlockSender#close we actually could IOE multiple times but
only increment once. Thoughts?
* Extra debug print in TestDataTransferKeepalive#testSlowReader
* Linebreak here is kind of awkward, move the end parens or the try up?
{code}
try (ReplicaHandler replica = dataset.append(
block, block.getGenerationStamp() + 1, block.getNumBytes())
) {
{code}
* TestMover adds a test timeout, looks unrelated to this patch?
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623649#comment-14623649
]
Lei (Eddy) Xu commented on HDFS-8694:
-
The checkstyle warnings are false alarm. They complained about the length of
the files.
The failure test {{TestStandbyIsHot}} is not relevent. I can not produce it on
my dev machine, and it seems to appear in other recent JIRAs.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623085#comment-14623085
]
Hadoop QA commented on HDFS-8694:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 15s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 8 new or modified test files. |
| {color:green}+1{color} | javac | 7m 37s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 20s | The applied patch generated 2
new checkstyle issues (total was 424, now 418). |
| {color:green}+1{color} | whitespace | 0m 4s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 20s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 30s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 2s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 162m 5s | Tests failed in hadoop-hdfs. |
| | | 205m 54s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 0824426 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11667/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11667/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11667/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11667/console |
This message was automatically generated.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614177#comment-14614177
]
Hadoop QA commented on HDFS-8694:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 47s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 8 new or modified test files. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 38s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 12s | The applied patch generated 2
new checkstyle issues (total was 446, now 440). |
| {color:green}+1{color} | whitespace | 0m 4s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 16s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 16s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 159m 34s | Tests failed in hadoop-hdfs. |
| | | 205m 52s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 688617d |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11575/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11575/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11575/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11575/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11575/console |
This message was automatically generated.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614369#comment-14614369
]
Lei (Eddy) Xu commented on HDFS-8694:
-
The failure test {{TestStandbyIsHot}} passed on my local machine. And the
checkstyle warnings are false alarm.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614132#comment-14614132
]
Hadoop QA commented on HDFS-8694:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 18m 5s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 8 new or modified test files. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 16s | The applied patch generated 2
new checkstyle issues (total was 446, now 440). |
| {color:green}+1{color} | whitespace | 0m 5s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 17s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 16s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 158m 48s | Tests failed in hadoop-hdfs. |
| | | 205m 35s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2eae130 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11574/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11574/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11574/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11574/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11574/console |
This message was automatically generated.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
您的邮件已收到！谢谢！
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606992#comment-14606992
]
Hadoop QA commented on HDFS-8694:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 22m 35s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 6 new or modified test files. |
| {color:red}-1{color} | javac | 9m 3s | The applied patch generated 2
additional warning messages. |
| {color:green}+1{color} | javadoc | 11m 22s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 27s | The applied patch generated 4
new checkstyle issues (total was 445, now 441). |
| {color:green}+1{color} | whitespace | 0m 3s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 12s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 16s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 177m 31s | Tests failed in hadoop-hdfs. |
| | | 232m 4s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.web.TestWebHdfsFileSystemContract |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| | hadoop.hdfs.TestDataTransferKeepalive |
| Timed out tests | org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
|
| | org.apache.hadoop.hdfs.server.mover.TestMover |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12742648/HDFS-8694.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / d3797f9 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
|
| javac |
https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/diffJavacWarnings.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11538/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11538/console |
This message was automatically generated.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
