[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910640#comment-16910640 ] Hudson commented on HDFS-14687: --- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17148 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17148/]) HDFS-14746. Trivial test code update after HDFS-14687. Contributed by (weichiu: rev abae6ff2a2760500b7e7d4414a43069ed4a45930) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestPendingDataNodeMessages.java > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Fix For: 3.0.4, 3.3.0, 3.2.1, 3.1.3 > > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch, HDFS-14687.004.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910111#comment-16910111 ] Hudson commented on HDFS-14687: --- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17145 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17145/]) HDFS-14687. Standby Namenode never come out of safemode when EC files (weichiu: rev b8db5b9a9812023754ed1b3e5b428e161f0add50) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingDataNodeMessages.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestPendingDataNodeMessages.java > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Fix For: 3.0.4, 3.3.0, 3.2.1, 3.1.3 > > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch, HDFS-14687.004.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910105#comment-16910105 ] Wei-Chiu Chuang commented on HDFS-14687: +1 Thanks. On my machine this test took 1.5 minutes. Longer than your number but much better than before. > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch, HDFS-14687.004.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909945#comment-16909945 ] Hadoop QA commented on HDFS-14687: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 46s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 3s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 10s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 29s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 18s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 30s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}154m 13s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestRollingUpgrade | \\ \\ || Subsystem || Report/Notes || | Docker | Client=18.09.7 Server=18.09.7 Image:yetus/hadoop:bdbca0e53b4 | | JIRA Issue | HDFS-14687 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12977884/HDFS-14687.004.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 7d915e48197d 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 3bba808 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_222 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/27551/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/27551/testReport/ | | Max. process+thread count | 2937 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/27551/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909926#comment-16909926 ] Surendra Singh Lilhore commented on HDFS-14687: --- Thanks [~jojochuang], Attached new patch, reduce number of datanode, earlier it was using 9 DN's (6+3 policy), now it is using 3 DN's (2+1 policy). In my machine it is taking 3 seconds. > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch, HDFS-14687.004.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909655#comment-16909655 ] Wei-Chiu Chuang commented on HDFS-14687: [~surendrasingh] I think the fix is good. But the test ran for more than 3 minutes on my machine. Can we update the test and cut down some wait time? It doesn't look like a large integration test. > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909649#comment-16909649 ] Wei-Chiu Chuang commented on HDFS-14687: Sorry [~surendrasingh] ididn't mean the patch itself is bad. I meant to say it's a bad bug. > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908047#comment-16908047 ] Siyao Meng commented on HDFS-14687: --- [~surendrasingh] rev 003 lgtm+1. Thanks! [~jojochuang] Any further comments? > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907935#comment-16907935 ] Hadoop QA commented on HDFS-14687: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 45s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 9s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 81m 36s{color} | {color:green} hadoop-hdfs in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 29s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}134m 29s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e | | JIRA Issue | HDFS-14687 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12977674/HDFS-14687.003.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 89e29826f879 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 85a00fd | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_222 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/27519/testReport/ | | Max. process+thread count | 5576 (vs. ulimit of 1) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/27519/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > Standby Namenode never come out of safemode when EC files are being written. > > > Key:
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907854#comment-16907854 ] Surendra Singh Lilhore commented on HDFS-14687: --- [~smeng] for review, attached updated patch. {quote} Looking at the fix, this looks really bad. {quote} [~jojochuang], do you have any other fix for this, I didn't get what is bad here ? > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch, > HDFS-14687.003.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907387#comment-16907387 ] Wei-Chiu Chuang commented on HDFS-14687: Ping [~surendrasingh] appreciate your report. Looking at the fix, this looks really bad. Could you help update the patch? Thank you > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903607#comment-16903607 ] Siyao Meng commented on HDFS-14687: --- [~surendrasingh] Great work. Thanks for the patch. The fix looks good to me. Unit test passed locally. Just two minor issues: 1. Please address the checkstyle [warnings|https://builds.apache.org/job/PreCommit-HDFS-Build/27409/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt] (line length). 2. Could you replace Thread.sleep(4000) in the test with either: (1) another method that doesn't rely on timing, like what is done in [HDFS-10240|https://github.com/apache/hadoop/commit/1290e3c647092f0bfbb250731a6805aba1be8e4b#diff-e069dc77becb87b909a07d27d3525236R1338]; (2) use a loop with small intervals (e.g. 100 ms) to check the status of DataNode IBR until it finishes; BTW I tried to remove Thread.sleep(4000) and ran the test. It passed as well (locally on my Mac). Is this sleep() call necessarily? Please remove if not. > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900494#comment-16900494 ] Hadoop QA commented on HDFS-14687: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 29s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 31s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 38s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 10 unchanged - 0 fixed = 12 total (was 10) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 53s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}103m 45s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}167m 33s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.TestLargeBlockReport | | | hadoop.hdfs.tools.TestDFSZKFailoverController | | | hadoop.hdfs.server.datanode.TestDataNodeMetrics | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 | | JIRA Issue | HDFS-14687 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12976751/HDFS-14687.002.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 212b42ae013d 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / d6697da | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_222 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/27409/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | whitespace |
[jira] [Commented] (HDFS-14687) Standby Namenode never come out of safemode when EC files are being written.
[ https://issues.apache.org/jira/browse/HDFS-14687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900371#comment-16900371 ] Surendra Singh Lilhore commented on HDFS-14687: --- Fixed failed test case.. > Standby Namenode never come out of safemode when EC files are being written. > > > Key: HDFS-14687 > URL: https://issues.apache.org/jira/browse/HDFS-14687 > Project: Hadoop HDFS > Issue Type: Bug > Components: ec, namenode >Affects Versions: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: Surendra Singh Lilhore >Priority: Critical > Attachments: HDFS-14687.001.patch, HDFS-14687.002.patch > > > When huge number of EC files are being written and SBN is restarted then it > will never come out of safe mode and required blocks count getting increase. > {noformat} > The reported blocks 16658401 needs additional 1702 blocks to reach the > threshold 0.9 of total blocks 16660120. > The reported blocks 16658659 needs additional 2935 blocks to reach the > threshold 0.9 of total blocks 16661611. > The reported blocks 16659947 needs additional 3868 blocks to reach the > threshold 0.9 of total blocks 16663832. > The reported blocks 1335 needs additional 5116 blocks to reach the > threshold 0.9 of total blocks 16671468. > The reported blocks 16669311 needs additional 6384 blocks to reach the > threshold 0.9 of total blocks 16675712. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org