haiyang1987 commented on PR #6464: URL: https://github.com/apache/hadoop/pull/6464#issuecomment-1903406926
> > This is a bug fix after #5564 , do you have time to help review this? > > @smarthanwang I have a question about [HDFS-16985](https://issues.apache.org/jira/browse/HDFS-16985), Normally FileNotFoundException means that the meta file or data file maybe lost, so the replication on this datanode maybe corrupt, right? In your business(AWS EC2 + EBS) situation, you don't expect datanode to delete this replica directly, so [HDFS-16985](https://issues.apache.org/jira/browse/HDFS-16985) just remove the replica from the memory of DN. > > But I want to see that DN should directly delete this corrupt replica If it can ensure that the replica is corrupt, such as: meta file or data file is lost. So we can add a configure to control whether DN delete this replication from disk directly, such as: fs.datanode.delete.corrupt.replica.from.disk with a default value true. > > If `fs.datanode.delete.corrupt.replica.from.disk` is true, DN can delete this corrupt replica from disk directly. If `fs.datanode.delete.corrupt.replica.from.disk` is false, DN can just delete this corrupt replica from memory. > > @smarthanwang @zhangshuyan0 looking forward to your good ideas. Thanks @ZanderXu for your comment. I agree with add new param to control whether this scenario requires deleting the replica from the disk. from the datanode side, if it is confirmed that the replica is not exists (meta file or data file is lost), it seems maybe reasonable that this replica should also be deleted (residual meta file or data file) from the disk. thanks~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
