tomscut commented on code in PR #6926:
URL: https://github.com/apache/hadoop/pull/6926#discussion_r1895316782
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java:
##########
@@ -3863,5 +3864,27 @@ public void setLastDirScannerFinishTime(long time) {
public long getPendingAsyncDeletions() {
return asyncDiskService.countPendingDeletions();
}
+
+ @Override
+ public void hardLinkOneBlock(ExtendedBlock srcBlock, ExtendedBlock dstBlock)
throws IOException {
+ BlockLocalPathInfo blpi = getBlockLocalPathInfo(srcBlock);
+ FsVolumeImpl v = getVolume(srcBlock);
+
+ try (AutoCloseableLock lock = lockManager.writeLock(LockLevel.VOLUME,
dstBlock.getBlockPoolId(),
+ v.getStorageID())) {
+ File src = new File(blpi.getBlockPath());
+ File srcMeta = new File(blpi.getMetaPath());
+ BlockPoolSlice dstBPS = v.getBlockPoolSlice(dstBlock.getBlockPoolId());
+ File dstBlockFile = dstBPS.hardLinkOneBlock(src, srcMeta,
dstBlock.getLocalBlock());
+
+ ReplicaInfo replicaInfo =
+ new LocalReplicaInPipeline(dstBlock.getBlockId(),
dstBlock.getGenerationStamp(), v,
+ dstBlockFile.getParentFile(),
dstBlock.getLocalBlock().getNumBytes());
+ dstBlockFile = dstBPS.addFinalizedBlock(dstBlock.getLocalBlock(),
replicaInfo);
+ replicaInfo = new FinalizedReplica(dstBlock.getLocalBlock(),
getVolume(srcBlock),
+ dstBlockFile.getParentFile());
+ volumeMap.add(dstBlock.getBlockPoolId(), replicaInfo);
Review Comment:
> dst volume write lock protects hardlink file atomicity and
volumeMap.add(). The best case scenario we should use src volume read lock and
dst volume write lock to protect src and dst file operation.
>
> And in DataXceiver, writeBlock() readBlock() transferBlock() , is as same
as this only require lock with block meta operation.
>
> So, I think this is enough for now. @Hexiaoqiao he @KeeProMise
Based on the offline discussion, I think this change is acceptable for now.
However, we need to restrict the scenario, such as the directory being updated
cannot use fastcopy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
