[
https://issues.apache.org/jira/browse/HDFS-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13710081#comment-13710081
]
Suresh Srinivas commented on HDFS-4942:
---------------------------------------
bq. Check the uniqueness guarantees though.
Every RPC client is uniquely identified. That means all the requests from a
single client can be identified as belonging to the same session.
bq. Is that a hypothetical opportunity or you have any particular use cases in
mind for Yarn? Would be good to know.
No. For stateful restarts of RM, this will be used to build retry cache for
non-idempotent request. Similarly other application masters such mapreduce and
Tez would make use of them.
bq. I mean that you are building a retry cache for HDFS and making changes
incompatible for all other projects (rather than for HDFS only). So I am trying
to understand what value it brings to others.
The above comment answers the value it brings to others. You keep referring to
incompatible changes. 2.1.0 is incompatible! This is going to to go with 2.1.0.
Also whether you make only HDFS incompatible or the entire Hadoop incompatible,
the applications need to deal with it anyway, given pretty much every one up
the stack uses HDFS.
> Add retry cache support in Namenode
> -----------------------------------
>
> Key: HDFS-4942
> URL: https://issues.apache.org/jira/browse/HDFS-4942
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha, namenode
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Attachments: HDFSRetryCache.pdf
>
>
> In current HA mechanism with FailoverProxyProvider and non HA setups with
> RetryProxy retry a request from the RPC layer. If the retried request has
> already been processed at the namenode, the subsequent attempts fail for
> non-idempotent operations such as create, append, delete, rename etc. This
> will cause application failures during HA failover, network issues etc.
> This jira proposes adding retry cache at the namenode to handle these
> failures. More details in the comments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
