[
https://issues.apache.org/jira/browse/HDFS-4308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540611#comment-13540611
]
Plamen Jeliazkov commented on HDFS-4308:
----------------------------------------
>From my inspection it seems that both getAdditionalBlock and abandonBlock
>write the blocks and path to editsLog twice. Both functions make a call to
>removeBlock, followed by persistBlocks. In fact, the only 2 removeBlock calls
>in FSNamesystem are both directly followed by persistBlocks.
removeBlock calls a legacy function which issues an AddOp to the editsLog,
writing the path, repl, mtime, atime, blocksize, blocks, perms, clientName, and
clientMachine. The call to persistBlocks writes only the path and blocks. It
appears to me that persistBlocks is the optimal way to go and therefore
removeBlocks should be altered.
We can optimize a little here by removing the call to write to the editsLog
inside removeBlock.
The 1-liner I will upload shortly removes the writing to the editsLog for
removeBlock. This is justified because it will be persisted by the call to
persistBlocks.
I have ran the Hadoop-HDFS tests following my patch and they all passed.
> addBlock() should persist file blocks once
> ------------------------------------------
>
> Key: HDFS-4308
> URL: https://issues.apache.org/jira/browse/HDFS-4308
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.0.3-alpha
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
>
> FSNamesystem.getAdditionalBlock() persists all file blocks twice, first in
> {{dir.removeBlock()}} using OP_ADD, then immediately after that in
> {{dir.persistBlocks()}} using OP_UPDATE_BLOCKS.
> This should be aggregated in one call to logEdit().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
