[
https://issues.apache.org/jira/browse/HADOOP-10597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HADOOP-10597:
-----------------------------
Attachment: HADOOP-10597.patch
Steve, thanks for the suggestions. Yes, it is useful to have server provide
backoff parameters to client.
Here is the initial patch to cover the basic idea. After we believe this is the
right direction, unit test and load tests will follow.
1. Enhance RetriableException for this purpose. When RPC queue is full,
RetriableException will be thrown back to the client with the backoff
parameters in PB. The backoff parameters are based on exponential back off.
2. Client/RetryPolicy can decide if and how to use the server hint. It is up to
each retry policy implementation. For NN HA RetryPolicy
failoverOnNetworkException, it will use the server hint when it is available.
For NN non-HA scenario, HDFS-6478 needs to be fixed first. A new policy called
RetryUpToMaximumCountWithFixedSleepAndServerHint is provided as an example.
3. Backoff feature is turned off by default in RPC server.
> Evaluate if we can have RPC client back off when server is under heavy load
> ---------------------------------------------------------------------------
>
> Key: HADOOP-10597
> URL: https://issues.apache.org/jira/browse/HADOOP-10597
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Ming Ma
> Attachments: HADOOP-10597.patch
>
>
> Currently if an application hits NN too hard, RPC requests be in blocking
> state, assuming OS connection doesn't run out. Alternatively RPC or NN can
> throw some well defined exception back to the client based on certain
> policies when it is under heavy load; client will understand such exception
> and do exponential back off, as another implementation of
> RetryInvocationHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
