[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.
[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-11087: -- Fix Version/s: (was: 2.9.0) > NamenodeFsck should check if the output writer is still writable. > - > > Key: HDFS-11087 > URL: https://issues.apache.org/jira/browse/HDFS-11087 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.6.5 >Reporter: Konstantin Shvachko >Assignee: Erik Krogen > Fix For: 2.8.0, 2.7.4 > > Attachments: HDFS-11087-branch-2.000.patch, > HDFS-11087-branch-2.001.patch, HDFS-11087.branch-2.000.patch > > > {{NamenodeFsck}} keeps running even after the client was interrupted. So if > you start {{fsck /}} on a large namespace and kill the client, the NameNode > will keep traversing the tree for hours although there is nobody to receive > the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.
[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-11087: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.4 2.9.0 2.8.0 Status: Resolved (was: Patch Available) I just committed this to branches 2, 2.8, 2.7, but not to trunk as it is not a problem there as noted earlier. Thank you [~xkrogen]. > NamenodeFsck should check if the output writer is still writable. > - > > Key: HDFS-11087 > URL: https://issues.apache.org/jira/browse/HDFS-11087 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.6.5 >Reporter: Konstantin Shvachko >Assignee: Erik Krogen > Fix For: 2.8.0, 2.9.0, 2.7.4 > > Attachments: HDFS-11087-branch-2.000.patch, > HDFS-11087-branch-2.001.patch, HDFS-11087.branch-2.000.patch > > > {{NamenodeFsck}} keeps running even after the client was interrupted. So if > you start {{fsck /}} on a large namespace and kill the client, the NameNode > will keep traversing the tree for hours although there is nobody to receive > the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.
[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11087: --- Attachment: HDFS-11087-branch-2.001.patch Thanks for the review, [~shv]. Attaching v001 patch with an additional comment. > NamenodeFsck should check if the output writer is still writable. > - > > Key: HDFS-11087 > URL: https://issues.apache.org/jira/browse/HDFS-11087 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.6.5 >Reporter: Konstantin Shvachko >Assignee: Erik Krogen > Attachments: HDFS-11087-branch-2.000.patch, > HDFS-11087-branch-2.001.patch, HDFS-11087.branch-2.000.patch > > > {{NamenodeFsck}} keeps running even after the client was interrupted. So if > you start {{fsck /}} on a large namespace and kill the client, the NameNode > will keep traversing the tree for hours although there is nobody to receive > the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.
[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11087: --- Attachment: HDFS-11087-branch-2.000.patch > NamenodeFsck should check if the output writer is still writable. > - > > Key: HDFS-11087 > URL: https://issues.apache.org/jira/browse/HDFS-11087 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.6.5 >Reporter: Konstantin Shvachko >Assignee: Erik Krogen > Attachments: HDFS-11087-branch-2.000.patch, > HDFS-11087.branch-2.000.patch > > > {{NamenodeFsck}} keeps running even after the client was interrupted. So if > you start {{fsck /}} on a large namespace and kill the client, the NameNode > will keep traversing the tree for hours although there is nobody to receive > the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.
[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11087: --- Attachment: HDFS-11087.branch-2.000.patch Attaching v000 patch. Since {{checkError}} internally calls {{flush}}, probably best not to do it _too_ frequently. We already flush after processing every 100 files, so I piggybacked off of that, replacing the call to {{flush}} with a call to {{checkError}} which will then throw an exception if the stream has been closed / otherwise failed. > NamenodeFsck should check if the output writer is still writable. > - > > Key: HDFS-11087 > URL: https://issues.apache.org/jira/browse/HDFS-11087 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.6.5 >Reporter: Konstantin Shvachko >Assignee: Erik Krogen > Attachments: HDFS-11087.branch-2.000.patch > > > {{NamenodeFsck}} keeps running even after the client was interrupted. So if > you start {{fsck /}} on a large namespace and kill the client, the NameNode > will keep traversing the tree for hours although there is nobody to receive > the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.
[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11087: --- Status: Patch Available (was: Open) > NamenodeFsck should check if the output writer is still writable. > - > > Key: HDFS-11087 > URL: https://issues.apache.org/jira/browse/HDFS-11087 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.6.5 >Reporter: Konstantin Shvachko >Assignee: Erik Krogen > > {{NamenodeFsck}} keeps running even after the client was interrupted. So if > you start {{fsck /}} on a large namespace and kill the client, the NameNode > will keep traversing the tree for hours although there is nobody to receive > the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.
[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-11087: --- Assignee: Erik Krogen > NamenodeFsck should check if the output writer is still writable. > - > > Key: HDFS-11087 > URL: https://issues.apache.org/jira/browse/HDFS-11087 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.6.5 >Reporter: Konstantin Shvachko >Assignee: Erik Krogen > > {{NamenodeFsck}} keeps running even after the client was interrupted. So if > you start {{fsck /}} on a large namespace and kill the client, the NameNode > will keep traversing the tree for hours although there is nobody to receive > the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org