[
https://issues.apache.org/jira/browse/HDFS-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730075#comment-14730075
]
Chris Douglas commented on HDFS-9020:
-------------------------------------
bq. caching dfsclients is nontrivial as the cache uses the instances of UGI,
but not the principals of it
Are you referring to the FileSystem cache? I haven't traced this, but the idea
was to map an incoming {{POST}} to the HDFS stream, not just the client
instance. But I may not catch your meaning... do you have a pointer into the
code?
bq. the client cannot recover once the stateful DN is down.
[~daryn] pointed out that we need a timeout for idle connections, which should
let us time out the connection and recover, at least to the extent we can in
the existing protocol. The existing implementation already relies on TCP
timeouts... and is intolerant of multiple failures, particularly for the append
case, right? I'm not dismissing the complexity of handling state, but is it
adding novel failure modes?
The HTTP/2 work (HDFS-7966) should dominate WebSockets, which is pretty raw.
The goal is shared: breaking up the stream into a sequence of {{\[PUT\]POST\*}}
operations is trying to be message-based. Similar to the point Todd and Stack
[made|https://issues.apache.org/jira/browse/HDFS-7966?focusedCommentId=14588913&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14588913],
a WebSockets protocol is yet-another thing to maintain...
> Support hflush/hsync in WebHDFS
> -------------------------------
>
> Key: HDFS-9020
> URL: https://issues.apache.org/jira/browse/HDFS-9020
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
> Reporter: Chris Douglas
> Attachments: HDFS-9020-alt.txt
>
>
> In the current implementation, hflush/hsync have no effect on WebHDFS
> streams, particularly w.r.t. visibility to other clients. This proposes to
> extend the protocol and implementation to enable this functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
