[
https://issues.apache.org/jira/browse/HDFS-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729975#comment-14729975
]
Haohui Mai commented on HDFS-9020:
----------------------------------
I think supporting hflush / hsync has a lot of values.
I think making WebHDFS stateful across connection is a bad idea, because (1)
caching dfsclients is nontrivial as the cache uses the instances of UGI, but
not the principals of it. (2) the client cannot recover once the stateful DN is
down.
The issue is that WebHDFS is a stream but not message-oriented protocol.
Without making it message-oriented I don't really see how hflush and hsync can
work when there are multiple failures in large clusters. Chunked write is a
good direction, but it might make sense to take a look at the websocket
protocol which provides you facilities for implementing chunked writes and
hflush / hsync messages. It will simplify things a lot.
> Support hflush/hsync in WebHDFS
> -------------------------------
>
> Key: HDFS-9020
> URL: https://issues.apache.org/jira/browse/HDFS-9020
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
> Reporter: Chris Douglas
> Attachments: HDFS-9020-alt.txt
>
>
> In the current implementation, hflush/hsync have no effect on WebHDFS
> streams, particularly w.r.t. visibility to other clients. This proposes to
> extend the protocol and implementation to enable this functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
