[
https://issues.apache.org/jira/browse/HDFS-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404260#comment-13404260
]
Todd Lipcon commented on HDFS-1108:
-----------------------------------
Hey Sanjay. I looked at the flush() vs hflush() change that you mentioned. It
looked like it changed between the attachment #12507073 and #12507492 on
HDFS-2602, so I talked to Aaron about it. We determined that he probably
changed it due to my review comment:
{quote}
What's the logic behind this? I think there are some cases in which there would
be multiple OP_ADDs for the same file with the same blocks. For example, the
client side hflush() call can trigger persistBlocks, but in the case of HA, the
last block would already have been persisted by getAdditionalBlock as well. So
the last block would be prematurely marked as COMPLETE in this case.
{quote}
So, I agree with you that we should change the hflush() back to flush(), but we
should also add another test case which does use hflush, so that we test the
case where the blocks are auto-logged due to the PERSIST_BLOCKS=true setting,
and then logged again due to hflush(). Perhaps we can add a flag to the test
like {{boolean useHflush}} and run it with both false and true. How does that
sound?
> Log newly allocated blocks
> --------------------------
>
> Key: HDFS-1108
> URL: https://issues.apache.org/jira/browse/HDFS-1108
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: dhruba borthakur
> Assignee: Todd Lipcon
> Fix For: HA branch (HDFS-1623)
>
> Attachments: HDFS-1108.patch, hdfs-1108-habranch.txt,
> hdfs-1108-habranch.txt, hdfs-1108-habranch.txt, hdfs-1108-habranch.txt,
> hdfs-1108-habranch.txt, hdfs-1108-hadoop-1-v2.patch,
> hdfs-1108-hadoop-1.patch, hdfs-1108.txt
>
>
> The current HDFS design says that newly allocated blocks for a file are not
> persisted in the NN transaction log when the block is allocated. Instead, a
> hflush() or a close() on the file persists the blocks into the transaction
> log. It would be nice if we can immediately persist newly allocated blocks
> (as soon as they are allocated) for specific files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
