[
https://issues.apache.org/jira/browse/HDFS-16286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17436476#comment-17436476
]
daimin commented on HDFS-16286:
-------------------------------
[~sodonnell] Thanks for your reply.
I have visited the README document and the code implementation, the EC
validator looks good enough to me. And in fact, if I had known this work has
been done, I would not paying time to re-work it out.
Our motive to build such a tool initially is because we had data corruption of
EC block group in our production environment and one big problem is that we
could not tell which files are good or bad. After we checked all the suspicious
files, we think this tool may be useful to those who are using EC too.
Could you please pay some time on reviewing the patch? Thanks a lot.
> Debug tool to verify the correctness of erasure coding on file
> --------------------------------------------------------------
>
> Key: HDFS-16286
> URL: https://issues.apache.org/jira/browse/HDFS-16286
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding, tools
> Affects Versions: 3.3.0, 3.3.1
> Reporter: daimin
> Assignee: daimin
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Block data in erasure coded block group may corrupt and the block meta
> (checksum) is unable to discover the corruption in some cases such as EC
> reconstruction, related issues are: HDFS-14768, HDFS-15186, HDFS-15240.
> In addition to HDFS-15759, there needs a tool to check erasure coded file
> whether any block group has data corruption in case of other conditions
> rather than EC reconstruction, or the feature HDFS-15759(validation during EC
> reconstruction) is not open(which is close by default now).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
