LeonGao91 commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917576109
@virajjasani Yes, I just did some more research. Depends on the sequence of
loading cache and constructing volume map, sometimes hardlink will just be
removed by datanode restart. So the corner case that needs to be tested here is
not guaranteed.
I think a less disrupting way to fix this is by controlling
"deleteDuplicateReplicas" value in the BlockPoolSlice. We can disable this
until the DirectoryScanner kicks in. Therefore we only need to add a
VisibleForTesting method in BlockPoolSlice
```
@VisibleForTesting
void setDeleteDuplicateReplicasForTesting(boolean value) {
deleteDuplicateReplicas = value;
}
```
And in the test, we can do this
```
// In the beginning
conf.setBoolean(
DFSConfigKeys.DFS_DATANODE_DUPLICATE_REPLICA_DELETION,
false);
...
// When testing DirectoryScanner should clean up the hardlink
DirectoryScanner scanner = new DirectoryScanner(
cluster.getDataNodes().get(0).getFSDataset(), conf);
FsVolumeImpl volume = (FsVolumeImpl)
cluster.getDataNodes().get(0).getFSDataset().getFsVolumeReferences().get(0);
volume.getBlockPoolSlice(volume.getBlockPoolList()[0]).setDeleteDuplicateReplicasForTesting(true);
scanner.start();
scanner.run();
```
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
