[ https://issues.apache.org/jira/browse/HDFS-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324607#comment-16324607 ]
Ajay Kumar commented on HDFS-12942: ----------------------------------- {quote}While cleaning up the new replica on failure, we call volume.onBlockFileDeletion which decrements the space used for the block on the new target volume. However I am not sure that the space used was incremented in the first place by copyReplicaToVolume. This may be a pre-existing bug actually.{quote} [~arpitagarwal],[~virajith] good catch. {code} try (AutoCloseableLock lock = datasetLock.acquire()) { // Increment numBlocks here as this block moved without knowing to BPS FsVolumeImpl volume = (FsVolumeImpl) newReplicaInfo.getVolume(); volume.incrNumBlocks(block.getBlockPoolId()); }{code} Seems in case of success we should increment the dfs used and no of blocks for new volume and decrement the same for old block. Currently we only increment no of blocks. Is this bug or intentional? [~arpitagarwal], thanks for offline discussion on this. In case of failure we should free up memory on disk. (will update patch for same) {quote}Adding to this, in patch v4, the generation stamp check in finalizeReplica is done after a call to v.addFinalizedBlock which seems wasted work if the check doesn't pass. Can the generation stamp check be done before the call to v.addFinalizedBlock?{quote} Agree,Will move check in start of finalize call. > Synchronization issue in FSDataSetImpl#moveBlock > ------------------------------------------------ > > Key: HDFS-12942 > URL: https://issues.apache.org/jira/browse/HDFS-12942 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Ajay Kumar > Assignee: Ajay Kumar > Attachments: HDFS-12942.001.patch, HDFS-12942.002.patch, > HDFS-12942.003.patch, HDFS-12942.004.patch > > > FSDataSetImpl#moveBlock works in following following 3 steps: > # first creates a new replicaInfo object > # calls finalizeReplica to finalize it. > # Calls removeOldReplica to remove oldReplica. > A client can potentially append to the old replica between step 1 and 2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org