[
https://issues.apache.org/jira/browse/HDFS-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481057#comment-13481057
]
Jan Kunigk commented on HDFS-4089:
----------------------------------
Hi Todd, I only have a small tenure on HDFS, which means I don't have a history
of why this feature was put there, but considering your comments it seems like
the reason is performance.
I personally also think syncing after every write is tremendous sacrifice to
make for durability, but since the flag is called SyncBehindWrites it implies
just that meaning.
The flag is enabled in DFSConfigKeys.java in a section called "HA related", so
it might mislead a user/admin into believing this actually enforces synchronous
IO.
I'm happy to close this, but I ideally the flags should be renamed to something
like WriteOutDirtyOSPages, since that is really what's done.
Thank you for your swift reply in any case.
> SyncBehindWrites uses wrong flags on sync_file_range
> ----------------------------------------------------
>
> Key: HDFS-4089
> URL: https://issues.apache.org/jira/browse/HDFS-4089
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, ha
> Reporter: Jan Kunigk
> Priority: Minor
> Attachments: syncBehindWrites.patch
>
>
> Hi, I stumbled upon this while trying to understand the append design
> recently. I am assuming when SyncBehindWrites is enabled we do indeed want to
> do a complete sync after each write. In that case the implementation seems
> wrong to me.
> Here's a comment from the manpage of sync_file_range on the usage of the
> SYNC_FILE_RANGE_WRITE flag in solitude: "This is an asynchronous
> flush-to-disk operation. This is not suitable for data integrity operations."
> I don't know why this syscall is invoked here instead of just fsync
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
