[
https://issues.apache.org/jira/browse/HDFS-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192370#comment-13192370
]
Aaron T. Myers commented on HDFS-2713:
--------------------------------------
bq. We cached the active connection in client in our internal version. So, that
in the same JVM, every new client need not perform failover.
Presumably with some static map or static reference to a client connection?
That sort of design terrifies me, with the potential for subtle bugs that are
tough to reproduce and write tests for. So, without really, really good
justification, I'd be opposed to such a change.
bq. Do you think this will help us to solve below case in this HA proposal(
atleast per JVM level) ?
Maybe, but only for a rather limited set of circumstances, i.e. when a single
JVM has multiple DFS clients, and a failover has occurred during the lifetime
of the JVM. It will do nothing for what I suspect will be a far more common
case - a brand new JVM (e.g. `hadoop fs -ls') that has exactly one client
connection to the NN, and has to try the first-listed NN before attempting a
connection to the second.
I honestly believe that optimizing client side failover time to this degree
will be mostly wasted effort. We already do a client failover immediately when
an error is detected (i.e. we don't retry to the first-listed NN at all) so the
slow-down should be largely unnoticeable for most users. Doing a client
failover once per client connection per NN failover event just isn't that big a
deal.
If you really want to try to optimize client failover, I strongly recommend you
look into implementing "speculative RPC," i.e. for a given RPC call, try
connecting to both in parallel and see which one returns first. Regardless of
which NN is the active, one NN should either throw a StandbyException or other
error, while the other NN will return a correct result, which the client
library can return to the DFSClient. This seems way safer, and has the
advantage of benefiting even the fresh JVM case.
> HA : An alternative approach to clients handling Namenode failover.
> --------------------------------------------------------------------
>
> Key: HDFS-2713
> URL: https://issues.apache.org/jira/browse/HDFS-2713
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, hdfs client
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Uma Maheswara Rao G
> Assignee: Uma Maheswara Rao G
>
> This is the approach for client failover which we adopted when we developed
> HA for Hadoop. I would like to propose thia approach for others to review &
> include in the HA implementation, if found useful.
> This is similar to the ConfiguredProxyProvider in the sense that the it takes
> the address of both the Namenodes as the input. The major differences I can
> see from the current implementation are
> 1) During failover, user threads can be controlled very accurately about *the
> time they wait for active namenode* to be available, awaiting the retry.
> Beyond this, the threads will not be made to wait; DFS Client will throw an
> Exception indicating that the operation has failed.
> 2) Failover happens in a seperate thread, not in the client application
> threads. The thread will keep trying to find the Active Namenode until it
> succeeds.
> 3) This also means that irrespective of whether the operation's RetryAction
> is RETRY_FAILOVER or FAIL, the user thread can trigger the client's failover.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
