[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219762#comment-16219762
]
Wei Yan commented on HDFS-11384:
--------------------------------
For the following changes in the diff, why we set dSec to 0 when "j >=
concurrentThreads", instead of "j <= concurrentThreads"? Did I miss anything
there? [~shv], [~zhaoyunjiong], [~zhz]
{code}
// Calculate delay in seconds for the next iteration
if(j >= concurrentThreads) {
dSec = 0;
} else if((j + 1) % BALANCER_NUM_RPC_PER_SEC == 0) {
dSec++;
}
{code}
> Add option for balancer to disperse getBlocks calls to avoid NameNode's
> rpc.CallQueueLength spike
> -------------------------------------------------------------------------------------------------
>
> Key: HDFS-11384
> URL: https://issues.apache.org/jira/browse/HDFS-11384
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover
> Affects Versions: 2.7.3
> Reporter: yunjiong zhao
> Assignee: Konstantin Shvachko
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: HDFS-11384-007.patch, HDFS-11384-branch-2.7.011.patch,
> HDFS-11384-branch-2.8.011.patch, HDFS-11384.001.patch, HDFS-11384.002.patch,
> HDFS-11384.003.patch, HDFS-11384.004.patch, HDFS-11384.005.patch,
> HDFS-11384.006.patch, HDFS-11384.008.patch, HDFS-11384.009.patch,
> HDFS-11384.010.patch, HDFS-11384.011.patch, balancer.day.png,
> balancer.week.png
>
>
> When running balancer on hadoop cluster which have more than 3000 Datanodes
> will cause NameNode's rpc.CallQueueLength spike. We observed this situation
> could cause Hbase cluster failure due to RegionServer's WAL timeout.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
