[
https://issues.apache.org/jira/browse/HDFS-16646?focusedWorklogId=786515&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-786515
]
ASF GitHub Bot logged work on HDFS-16646:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Jun/22 11:05
Start Date: 30/Jun/22 11:05
Worklog Time Spent: 10m
Work Description: ZanderXu opened a new pull request, #4519:
URL: https://github.com/apache/hadoop/pull/4519
### Description of PR
As we all known, `StaticRouterRpcFairnessPolicyController` is very helpfully
for RBF to minimize impact of clients connecting to healthy vs unhealthy
nameNodes.
But in prod environment, the traffic of clients accessing each NS and the
pressure of downstream namenodes are dynamically changed. So if we only have
one static permit conf, RBF cannot able to adapt to the changes in traffic to
achieve optimal results.
So here I propose an elastic RouterRpcFairnessPolicyController to help RBF
adapt to traffic changes to achieve an optimal result.
The overall idea is:
- Each name service can configured the exclusive permits like
`StaticRouterRpcFairnessPolicyController`
- TotalPermits is more than sum(NsExclusivePermit) and mark TotalPermits -
sum(NsExclusivePermit) as SharedPermits
- Each name service can properly preempt the SharedPermits after it's own
exclusive permits is used up.
- But the maximum value of SharedPermits preempted by each nameservice
should be limited. Such as 20% of SharedPermits.
Suppose we have 200 handlers and 5 name services, and each name services
configured different exclusive Permits, like:
| NS1 | NS2 | NS3 | NS4 | NS5 | Concurrent NS |
|
Issue Time Tracking
-------------------
Worklog Id: (was: 786515)
Remaining Estimate: 0h
Time Spent: 10m
> [RBF] Improved isolation for downstream name nodes. {Elastic}
> -------------------------------------------------------------
>
> Key: HDFS-16646
> URL: https://issues.apache.org/jira/browse/HDFS-16646
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
