[
https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279619#comment-13279619
]
Lars Hofhansl commented on HDFS-744:
------------------------------------
Upon further inspection it turns out the issue with the metrics not being
updated in time is due to the fact the flushOrSync is not on the synchronous
hsync or hflush path in BlockReceiver.receivePacket.
When hsync returns the DNs have all gotten the data and the request to flush
and sync, but they are not necessarily finished with it.
The same, BTW, is currently (without my patch) true to hflush. When hflush
returns the DNs have received the data in their buffers, but they may not be
done flushing the data to the OS buffers.
If we wanted to fix this (which would involve doing the flushOrSync in the
PacketResponder if the request is a client request, which inconveniently would
also serialize the flush/syncs across replicas), this should be a different
jira, I think.
For now, to make the tests a bit more predictable I'll increment the sync
counter before the actual sync operation rather than afterwards.
> Support hsync in HDFS
> ---------------------
>
> Key: HDFS-744
> URL: https://issues.apache.org/jira/browse/HDFS-744
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: data-node, hdfs client
> Reporter: Hairong Kuang
> Assignee: Lars Hofhansl
> Attachments: HDFS-744-trunk-v2.patch, HDFS-744-trunk-v3.patch,
> HDFS-744-trunk-v4.patch, HDFS-744-trunk-v5.patch, HDFS-744-trunk-v6.patch,
> HDFS-744-trunk.patch, hdfs-744-v2.txt, hdfs-744-v3.txt, hdfs-744.txt
>
>
> HDFS-731 implements hsync by default as hflush. As descriibed in HADOOP-6313,
> the real expected semantics should be "flushes out to all replicas and all
> replicas have done posix fsync equivalent - ie the OS has flushed it to the
> disk device (but the disk may have it in its cache)." This jira aims to
> implement the expected behaviour.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
