[
https://issues.apache.org/jira/browse/HDFS-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174148#comment-13174148
]
Uma Maheswara Rao G commented on HDFS-2713:
-------------------------------------------
While writing a file if Active Namenode goes down, the thread cannot wait
forever until the Standby Namenode gets fully functional as Active. We place a
maximum limit on how long the user thread can wait for a Namenode to be
available. For futher discussions, can term this limit as namenodeAwaitTimeout.
If Namenode is not available even after namenodeAwaitTimeout, DFS Client will
throw an Exception back to the caller.
Corresponding to Namenode pair, there is a HASwitchAgent which is responsible
for managing connections to multiple Namenode addresses configured. The
HASwitchAgent exposes a performFailover() API, to trigger the switching logic.
During any DFS operation, if RetryInvocationHandler finds that Active Namenode
is down, the performFailover() API will be invoked. Then the HASwitchAgent will
execute the switching procedure in a seperate thread, by trying to connect to
the two Namenode addresses configured. This search for finding the Active
Namenode will continue until it successfully connects to an Active Namenode.
The thread which invoked the performFailover() API will go to a TIMED WAIT and
will be notified when the Active Namenode is available. The WAIT will happen
for the time configured as 'namenode.await.timeout'. Once this time has
elapsed, an IOException will be thrown from performFailover() API of
HASwitchAgent. The subsequent invocations of the same API may already find that
the switching logic is in progress and just make the calling thread to wait for
'namenode.await.timeout'.
> HA : An alternative approach to clients handling Namenode failover.
> --------------------------------------------------------------------
>
> Key: HDFS-2713
> URL: https://issues.apache.org/jira/browse/HDFS-2713
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, hdfs client
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Uma Maheswara Rao G
> Assignee: Uma Maheswara Rao G
>
> This is the approach for client failover which we adopted when we developed
> HA for Hadoop. I would like to propose thia approach for others to review &
> include in the HA implementation, if found useful.
> This is similar to the ConfiguredProxyProvider in the sense that the it takes
> the address of both the Namenodes as the input. The major differences I can
> see from the current implementation are
> 1) During failover, user threads can be controlled very accurately about *the
> time they wait for active namenode* to be available, awaiting the retry.
> Beyond this, the threads will not be made to wait; DFS Client will throw an
> Exception indicating that the operation has failed.
> 2) Failover happens in a seperate thread, not in the client application
> threads. The thread will keep trying to find the Active Namenode until it
> succeeds.
> 3) This also means that irrespective of whether the operation's RetryAction
> is RETRY_FAILOVER or FAIL, the user thread can trigger the client's failover.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
