Xiao Chen updated HDFS-11009:
    Attachment: HDFS-11009.03.patch

Thanks Andrew for reviewing! Patch 3 to address most comments, with some 

bq. It's also debatable whether these should be flags at all, since UNIX 
normally uses positional args if they're required.
Good point. I'm keeping it for consistency with other {{debug}} commands: 
{{-meta}} for {{verifyMeta}} and {{-path}} for {{recoverLease}} are flagged. 
Please let me know if you feel strongly.

bq. I'm guessing this is meant for use in emergency situations where the block 
only has corrupt replicas? I'm also wondering about the danger of divergent 
state on different DNs, since I don't think HDFS-level mechanisms like the 
BlockScanner or FBRs will catch this.
Exactly. So if there's no good replicas, but the HDFS file is critical, and the 
blockfile is not deleted on disk, there's a chance to restore the .meta file. 
One still need to search in the logs carefully to find genStamp, but it's 
theoretically doable.
I don't think BlockScanner would catch this and work, but I tried restarting DN 
locally (in a single DN env), and after FBR the block is readable again. It's 
not a 'divergent state' in this case, just DNs with corrupt replicas and good 
replicas, right?

> Add a tool to reconstruct block meta file from CLI
> --------------------------------------------------
>                 Key: HDFS-11009
>                 URL: https://issues.apache.org/jira/browse/HDFS-11009
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>         Attachments: HDFS-11009.01.patch, HDFS-11009.02.patch, 
> HDFS-11009.03.patch
> If the block file on local disk presents, but the meta file is missing, it's 
> theoretically possible to manually restore the meta file and have the block 
> readable.
> This jira is to propose to add such a tool to do that.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to