[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532426#comment-14532426
]
Hudson commented on HDFS-7833:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #187 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/187/])
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
required, based on configured failed volumes tolerated. Contributed by Lei
(Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532742#comment-14532742
]
Hudson commented on HDFS-7833:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2118 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2118/])
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
required, based on configured failed volumes tolerated. Contributed by Lei
(Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532890#comment-14532890
]
Hudson commented on HDFS-7833:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2136 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2136/])
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
required, based on configured failed volumes tolerated. Contributed by Lei
(Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532774#comment-14532774
]
Hudson commented on HDFS-7833:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #177 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/177/])
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
required, based on configured failed volumes tolerated. Contributed by Lei
(Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532816#comment-14532816
]
Hudson commented on HDFS-7833:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #187 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/187/])
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
required, based on configured failed volumes tolerated. Contributed by Lei
(Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532439#comment-14532439
]
Hudson commented on HDFS-7833:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #920 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/920/])
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
required, based on configured failed volumes tolerated. Contributed by Lei
(Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531135#comment-14531135
]
Chris Nauroth commented on HDFS-7833:
-
Hello [~eddyxu]. I apologize for the delay. Unfortunately, the patch needs a
rebase now. Would you please upload a v003? I'll prioritize reviewing it.
The new test looks like what I had in mind. I'll review in detail after the
rebase. Thanks for incorporating the feedback.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531755#comment-14531755
]
Hadoop QA commented on HDFS-7833:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 36s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 27s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 38s | The applied patch generated 2
new checkstyle issues (total was 275, now 275). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 6s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 166m 56s | Tests failed in hadoop-hdfs. |
| | | 208m 13s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.tracing.TestTraceAdmin |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12730951/HDFS-7833.003.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 185e63a |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10836/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10836/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10836/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10836/console |
This message was automatically generated.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531984#comment-14531984
]
Hudson commented on HDFS-7833:
--
FAILURE: Integrated in Hadoop-trunk-Commit #7754 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7754/])
HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
required, based on configured failed volumes tolerated. Contributed by Lei
(Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532064#comment-14532064
]
Lei (Eddy) Xu commented on HDFS-7833:
-
Thanks a lot for reviewing and committing this, [~cnauroth]!
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Labels: BB2015-05-TBR
Fix For: 2.8.0
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch, HDFS-7833.003.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527498#comment-14527498
]
Lei (Eddy) Xu commented on HDFS-7833:
-
Hey, [~cnauroth]. Would you mind to take a look at this JIRA? The above test
failure is not relevant.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512132#comment-14512132
]
Hadoop QA commented on HDFS-7833:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 36s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 29s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 7m 25s | There were no new checkstyle
issues. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 4s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | native | 3m 13s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 165m 10s | Tests failed in hadoop-hdfs. |
| | | 213m 6s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12728032/HDFS-7833.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 5ce3a77 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10378/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10378/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10378/console |
This message was automatically generated.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch,
HDFS-7833.002.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370812#comment-14370812
]
Hadoop QA commented on HDFS-7833:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705796/HDFS-7833.001.patch
against trunk revision e37ca22.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
org.apache.hadoop.tracing.TestTracing
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9995//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9995//console
This message is automatically generated.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370814#comment-14370814
]
Lei (Eddy) Xu commented on HDFS-7833:
-
These failing tests are not related.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7833.000.patch, HDFS-7833.001.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370321#comment-14370321
]
Chris Nauroth commented on HDFS-7833:
-
Hi [~eddyxu]. Thank you for the patch. Overall, this looks correct to me.
When I I filed this issue, it was before some of the discussion we had in
HDFS-7722, and I had 2 cases in mind that could trigger this bug:
* Admin reconfigures DataNode to remove a path that has a failed volume. As
per discussion in HDFS-7722, we've made the decision that this case should not
clear volume failure information. Since this is logically still considered a
volume failure, there is no harm done to the check for sufficient resources.
IOW, after the discussion in HDFS-7722, we don't have to worry about this case
anymore.
* Admin reconfigures DataNode and adds a few new paths that weren't there
before. This case is still a problem.
To properly cover the second case, let's add a test that does something like
this:
# Start a DataNode with 2 volumes and {{dfs.datanode.failed.volumes.tolerated}}
set to 1.
# Run DataNode reconfiguration to add a new volume. Now we're up to 3 volumes
total.
# Fail a volume. Assert that the DataNode continues running.
# Fail another volume. Assert that the DataNode stops running.
Without your patch, I expect this test would fail on the last step, because
{{validVolsRequired}} would have been calculated as 1, and we still have 1
volume remaining. After applying your patch, I expect the test would then pass.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7833.000.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370210#comment-14370210
]
Hadoop QA commented on HDFS-7833:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705686/HDFS-7833.000.patch
against trunk revision 61a4c7f.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestLeaseRecovery2
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9983//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9983//console
This message is automatically generated.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7833.000.patch
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366196#comment-14366196
]
Lei (Eddy) Xu commented on HDFS-7833:
-
Hi, [~cnauroth], thanks a lot for reporting this issue. I am trying to
understand this bug. It seems to me that if we follow the same old logic, we
need also update {{FsDdatasetImpl#validVolsRequired()}} when it removes a
directory _intentionly_ , but do not update it when the {{FsVolumeImpl}} is
removed due to {{checkDirs}}. As an alternative, since now failure volumes are
kept in {{FsDatasetImp#volumeFailureInfos}}, can we just let
{{FsDatasetImpl#hasEnoughResources()}} become:
{code}
public boolean hasEnoughResource() {
return volumeFailureInfos.size() volFailuresTolerated;
}
{code}
Does it make sense to you, [~cnauroth]?
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7833) DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
[
https://issues.apache.org/jira/browse/HDFS-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335228#comment-14335228
]
Chris Nauroth commented on HDFS-7833:
-
This is a repeat of the comment I mentioned on HDFS-7830. Thank you to
[~eddyxu] for volunteering to take assignment of the issue.
Another potential problem that I've noticed in the DataNode reconfiguration
code is that it never recalculates {{FsDatasetImpl#validVolsRequired}}. This is
a final variable calculated as (# volumes configured) - (# volume failures
tolerated):
{code}
this.validVolsRequired = volsConfigured - volFailuresTolerated;
{code}
If this variable is not updated for DataNode reconfigurations, then it could
lead to some unexpected situations. For example:
# DataNode starts running with 6 volumes (all healthy) and
{{dfs.datanode.failed.volumes.tolerated}} set to 2.
# {{FsDatasetImpl#validVolsRequired}} is set to 6 - 2 = 4.
# DataNode is reconfigured to run with 8 volumes (all still healthy).
# Now 3 volumes fail. The admin would expect the DataNode to abort, but there
are 8 - 3 = 5 good volumes left, and {{FsDatasetImpl#validVolsRequired}} is
still 4, so {{FsDatasetImpl#hasEnoughResource}} returns true.
DataNode reconfiguration does not recalculate valid volumes required, based
on configured failed volumes tolerated.
---
Key: HDFS-7833
URL: https://issues.apache.org/jira/browse/HDFS-7833
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Chris Nauroth
Assignee: Lei (Eddy) Xu
DataNode reconfiguration never recalculates
{{FsDatasetImpl#validVolsRequired}}. This may cause incorrect behavior of
the {{dfs.datanode.failed.volumes.tolerated}} property if reconfiguration
causes the DataNode to run with a different total number of volumes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
