[
https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227058#comment-15227058
]
Mingliang Liu commented on HDFS-7661:
-------------------------------------
Thanks for proposing solutions to this problem.
I agree with [~jingzhao] that there are still challenges that should be
addressed even if we keep non-full cellBuffers on the "parity DN". Moreover,
for different stripe, we have to maintain multiple cellBuffers on the "parity
DN" side as the "append" operation only happens in the same stripe. If the
writer fails during the hflush operation, we have to make sure last flushed
cellBuffer is still available, which may be in a previous (different) stripe.
If we manage all cellBuffers across different stripes in one file, we also need
some kind of "undo" mechanism for rolling back to last flushed data to handle
failure. Otherwise, managing those individual files may be nontrivial.
> [umbrella] support hflush and hsync for erasure coded files
> -----------------------------------------------------------
>
> Key: HDFS-7661
> URL: https://issues.apache.org/jira/browse/HDFS-7661
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: erasure-coding
> Reporter: Tsz Wo Nicholas Sze
> Assignee: GAO Rui
> Attachments: EC-file-flush-and-sync-steps-plan-2015-12-01.png,
> HDFS-7661-unitTest-wip-trunk.patch, HDFS-7661-wip.01.patch,
> HDFS-EC-file-flush-sync-design-v20160323.pdf,
> HDFS-EC-file-flush-sync-design-version1.1.pdf
>
>
> We also need to support hflush/hsync and visible length.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
