[
https://issues.apache.org/jira/browse/HADOOP-10597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171966#comment-14171966
]
Ming Ma commented on HADOOP-10597:
----------------------------------
Thanks, Chris.
1. The first two experiments belong to "user A is doing some bad things,
measure user B's read latency.". The rest of the experiments are done under
single user to measure the performance implication under different loads.
2. We can use client backoff without FCQ. But it is less interesting, given it
could penalize good clients. That is because in the current implementation, the
criteria RPC server uses to decide if it needs to ask client to back off is
whether the RPC call queue is full. We can improve this criteria later if this
isn't enough.
3. The experiment results are based on "client driven retry interval" policy.
It means the server only asks the client to back off; RPC client will decide
retry policy. In NN HA setup, that will be FailoverOnNetworkExceptionRetry
which does exponential back off.
> Evaluate if we can have RPC client back off when server is under heavy load
> ---------------------------------------------------------------------------
>
> Key: HADOOP-10597
> URL: https://issues.apache.org/jira/browse/HADOOP-10597
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Ming Ma
> Assignee: Ming Ma
> Attachments: HADOOP-10597-2.patch, HADOOP-10597.patch,
> MoreRPCClientBackoffEvaluation.pdf, RPCClientBackoffDesignAndEvaluation.pdf
>
>
> Currently if an application hits NN too hard, RPC requests be in blocking
> state, assuming OS connection doesn't run out. Alternatively RPC or NN can
> throw some well defined exception back to the client based on certain
> policies when it is under heavy load; client will understand such exception
> and do exponential back off, as another implementation of
> RetryInvocationHandler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
