[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993672#comment-15993672
]
Andrew Wang commented on HDFS-9922:
---
We need to keep the 2.9.0 fix version, since only major and minor versions are
supersets.
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
> --
>
> Key: HDFS-9922
> URL: https://issues.apache.org/jira/browse/HDFS-9922
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
>Priority: Minor
> Fix For: 2.9.0, 3.0.0-alpha1, 2.8.2
>
> Attachments: HDFS-9922.branch-2.8.001.patch,
> HDFS-9922-trunk-v1.patch, HDFS-9922-trunk-v2.patch, HDFS-9922-trunk-v3.patch,
> HDFS-9922-trunk-v4.patch
>
>
> When there are replicas of a block on a decommissioned node,
> BlockPlacementStatusWithUpgradeDomain#isUpgradeDomainPolicySatisfied returns
> false when it should return true. This is because numberOfReplicas is the
> number of in-service replicas for the block and upgradeDomains.size() is the
> number of upgrade domains across all replicas of the block. Specifically, we
> hit this scenario when numberOfReplicas is equal to upgradeDomainFactor and
> upgradeDomains.size() is greater than numberOfReplicas.
> {code}
> private boolean isUpgradeDomainPolicySatisfied() {
> if (numberOfReplicas <= upgradeDomainFactor) {
> return (numberOfReplicas == upgradeDomains.size());
> } else {
> return upgradeDomains.size() >= upgradeDomainFactor;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993580#comment-15993580
]
Hadoop QA commented on HDFS-9922:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
57s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
23s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 56m 45s{color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_121. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_121 Failed junit tests |
hadoop.hdfs.server.datanode.TestFsDatasetCache |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:5970e82 |
| JIRA Issue | HDFS-9922 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12866009/HDFS-9922.branch-2.8.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 94628fcd32db 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.8 / 305a9d8 |
| Default Java | 1.7.0_121
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979473#comment-15979473
]
Ming Ma commented on HDFS-9922:
---
Currently upgrade domain isn't considered available in 2.8 due to these
changes. If we want the feature to be in 2.8, the major backport item is
HDFS-9005.
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
> --
>
> Key: HDFS-9922
> URL: https://issues.apache.org/jira/browse/HDFS-9922
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
>Priority: Minor
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: HDFS-9922-trunk-v1.patch, HDFS-9922-trunk-v2.patch,
> HDFS-9922-trunk-v3.patch, HDFS-9922-trunk-v4.patch
>
>
> When there are replicas of a block on a decommissioned node,
> BlockPlacementStatusWithUpgradeDomain#isUpgradeDomainPolicySatisfied returns
> false when it should return true. This is because numberOfReplicas is the
> number of in-service replicas for the block and upgradeDomains.size() is the
> number of upgrade domains across all replicas of the block. Specifically, we
> hit this scenario when numberOfReplicas is equal to upgradeDomainFactor and
> upgradeDomains.size() is greater than numberOfReplicas.
> {code}
> private boolean isUpgradeDomainPolicySatisfied() {
> if (numberOfReplicas <= upgradeDomainFactor) {
> return (numberOfReplicas == upgradeDomains.size());
> } else {
> return upgradeDomains.size() >= upgradeDomainFactor;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978923#comment-15978923
]
Kihwal Lee commented on HDFS-9922:
--
Don't we need this in 2.8?
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
> --
>
> Key: HDFS-9922
> URL: https://issues.apache.org/jira/browse/HDFS-9922
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
>Priority: Minor
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: HDFS-9922-trunk-v1.patch, HDFS-9922-trunk-v2.patch,
> HDFS-9922-trunk-v3.patch, HDFS-9922-trunk-v4.patch
>
>
> When there are replicas of a block on a decommissioned node,
> BlockPlacementStatusWithUpgradeDomain#isUpgradeDomainPolicySatisfied returns
> false when it should return true. This is because numberOfReplicas is the
> number of in-service replicas for the block and upgradeDomains.size() is the
> number of upgrade domains across all replicas of the block. Specifically, we
> hit this scenario when numberOfReplicas is equal to upgradeDomainFactor and
> upgradeDomains.size() is greater than numberOfReplicas.
> {code}
> private boolean isUpgradeDomainPolicySatisfied() {
> if (numberOfReplicas <= upgradeDomainFactor) {
> return (numberOfReplicas == upgradeDomains.size());
> } else {
> return upgradeDomains.size() >= upgradeDomainFactor;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333114#comment-15333114
]
Hudson commented on HDFS-9922:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #9965 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9965/])
HDFS-9922. Upgrade Domain placement policy status marks a good block in
(mingma: rev b48f27e794e42ba90836314834e872616437d7c9)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementStatusWithUpgradeDomain.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockPlacementStatusWithUpgradeDomain.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestUpgradeDomainBlockPlacementPolicy.java
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
> --
>
> Key: HDFS-9922
> URL: https://issues.apache.org/jira/browse/HDFS-9922
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
>Priority: Minor
> Fix For: 2.9.0
>
> Attachments: HDFS-9922-trunk-v1.patch, HDFS-9922-trunk-v2.patch,
> HDFS-9922-trunk-v3.patch, HDFS-9922-trunk-v4.patch
>
>
> When there are replicas of a block on a decommissioned node,
> BlockPlacementStatusWithUpgradeDomain#isUpgradeDomainPolicySatisfied returns
> false when it should return true. This is because numberOfReplicas is the
> number of in-service replicas for the block and upgradeDomains.size() is the
> number of upgrade domains across all replicas of the block. Specifically, we
> hit this scenario when numberOfReplicas is equal to upgradeDomainFactor and
> upgradeDomains.size() is greater than numberOfReplicas.
> {code}
> private boolean isUpgradeDomainPolicySatisfied() {
> if (numberOfReplicas <= upgradeDomainFactor) {
> return (numberOfReplicas == upgradeDomains.size());
> } else {
> return upgradeDomains.size() >= upgradeDomainFactor;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333044#comment-15333044
]
Hadoop QA commented on HDFS-9922:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 27s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
6s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 6s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s {color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 55m 1s {color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 29s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestAsyncHDFSWithHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:e2f6409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12810880/HDFS-9922-trunk-v4.patch
|
| JIRA Issue | HDFS-9922 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux c393c9047340 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5dfc38f |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/15791/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HDFS-Build/15791/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15791/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15791/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
>
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332655#comment-15332655
]
Lei (Eddy) Xu commented on HDFS-9922:
-
+1 pending jenkins. Looks good to me. Thanks for the works here, [~ming] and
[~ctrezzo]
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
> --
>
> Key: HDFS-9922
> URL: https://issues.apache.org/jira/browse/HDFS-9922
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
>Priority: Minor
> Attachments: HDFS-9922-trunk-v1.patch, HDFS-9922-trunk-v2.patch,
> HDFS-9922-trunk-v3.patch, HDFS-9922-trunk-v4.patch
>
>
> When there are replicas of a block on a decommissioned node,
> BlockPlacementStatusWithUpgradeDomain#isUpgradeDomainPolicySatisfied returns
> false when it should return true. This is because numberOfReplicas is the
> number of in-service replicas for the block and upgradeDomains.size() is the
> number of upgrade domains across all replicas of the block. Specifically, we
> hit this scenario when numberOfReplicas is equal to upgradeDomainFactor and
> upgradeDomains.size() is greater than numberOfReplicas.
> {code}
> private boolean isUpgradeDomainPolicySatisfied() {
> if (numberOfReplicas <= upgradeDomainFactor) {
> return (numberOfReplicas == upgradeDomains.size());
> } else {
> return upgradeDomains.size() >= upgradeDomainFactor;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330899#comment-15330899
]
Hadoop QA commented on HDFS-9922:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs: The patch generated 10
new + 9 unchanged - 2 fixed = 19 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 56m 25s {color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 52s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12810679/HDFS-9922-trunk-v2.patch
|
| JIRA Issue | HDFS-9922 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 08289f722c13 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c77a109 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/15772/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/15772/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HDFS-Build/15772/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15772/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15772/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Upgrade
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330646#comment-15330646
]
Chris Trezzo commented on HDFS-9922:
Also, thank you [~mingma] for the review and help on unit tests!
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
> --
>
> Key: HDFS-9922
> URL: https://issues.apache.org/jira/browse/HDFS-9922
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
>Priority: Minor
> Attachments: HDFS-9922-trunk-v1.patch, HDFS-9922-trunk-v2.patch
>
>
> When there are replicas of a block on a decommissioned node,
> BlockPlacementStatusWithUpgradeDomain#isUpgradeDomainPolicySatisfied returns
> false when it should return true. This is because numberOfReplicas is the
> number of in-service replicas for the block and upgradeDomains.size() is the
> number of upgrade domains across all replicas of the block. Specifically, we
> hit this scenario when numberOfReplicas is equal to upgradeDomainFactor and
> upgradeDomains.size() is greater than numberOfReplicas.
> {code}
> private boolean isUpgradeDomainPolicySatisfied() {
> if (numberOfReplicas <= upgradeDomainFactor) {
> return (numberOfReplicas == upgradeDomains.size());
> } else {
> return upgradeDomains.size() >= upgradeDomainFactor;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189448#comment-15189448
]
Ming Ma commented on HDFS-9922:
---
Thanks [~ctrezzo]. The patch makes sense.
* Based on how it is {{verifyBlockPlacement}} is used, it seems
{{numOfReplicas}} means replication factor. Maybe we should modify the
parameter name {{numOfReplicas}} to {{replicationFactor}} at the
{{BlockPlacementPolicy}} level?
* There is another case. Say the replication factor is 2, the block has 3
replications temporarily due to over replication such as recommission. In such
case, {{numberOfReplicas}} is 2 and {{upgradeDomains.size()}} is 3. Wonder if
we can change the check from {{numberOfReplicas == upgradeDomains.size()}} to
{{numberOfReplicas <= upgradeDomains.size()}}.
* The new test case is good. But it passes even without the fix. So it will be
good if we can improve the test.
> Upgrade Domain placement policy status marks a good block in violation when
> there are decommissioned nodes
> --
>
> Key: HDFS-9922
> URL: https://issues.apache.org/jira/browse/HDFS-9922
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
>Priority: Minor
> Attachments: HDFS-9922-trunk-v1.patch
>
>
> When there are replicas of a block on a decommissioned node,
> BlockPlacementStatusWithUpgradeDomain#isUpgradeDomainPolicySatisfied returns
> false when it should return true. This is because numberOfReplicas is the
> number of in-service replicas for the block and upgradeDomains.size() is the
> number of upgrade domains across all replicas of the block. Specifically, we
> hit this scenario when numberOfReplicas is equal to upgradeDomainFactor and
> upgradeDomains.size() is greater than numberOfReplicas.
> {code}
> private boolean isUpgradeDomainPolicySatisfied() {
> if (numberOfReplicas <= upgradeDomainFactor) {
> return (numberOfReplicas == upgradeDomains.size());
> } else {
> return upgradeDomains.size() >= upgradeDomainFactor;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes
[
https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186555#comment-15186555
]
Hadoop QA commented on HDFS-9922:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 15s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 4s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 4s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 15s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 21s
{color} | {color:red} Patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 103m 15s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Failed junit tests |
hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.TestCrcCorruption |
| | hadoop.hdfs.TestFileAppend |
| | hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality |
| JDK v1.8.0_74 Timed out junit tests |
org.apache.hadoop.hdfs.TestWriteReadStripedFile |
| | org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs |
| | org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs |
| | org.apache.hadoop.hdfs.TestHFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
