[
https://issues.apache.org/jira/browse/HDFS-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Lapan updated HDFS-4420:
----------------------------
Attachment: HDFS-4420-v4.patch
New optimized version of patch. Now, full list of blocks to exclude built only
on first iteration. On all further iterations, we only add newly added blocks
(by timestamp).
This reduces exclude list construction by a factor of 2-10, depending on
exclude subtree size.
On our cluster with 800TB /hbase, we now have this process take 10 seconds
instead of 15 minutes.
> Provide a way to exclude subtree from balancing process
> -------------------------------------------------------
>
> Key: HDFS-4420
> URL: https://issues.apache.org/jira/browse/HDFS-4420
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer
> Reporter: Max Lapan
> Priority: Minor
> Attachments: Balancer-exclude-subtree-0.90.2.patch,
> Balancer-exclude-trunk.patch, Balancer-exclude-trunk-v2.patch,
> Balancer-exclude-trunk-v3.patch, HDFS-4420-v4.patch
>
>
> During balancer operation, it balances all blocks, regardless of their
> filesystem hierarchy. Sometimes, it would be usefull to exclude some subtree
> from balancing process.
> For example, regionservers data locality is cruical for HBase performance.
> Region's data is tied to regionservers, which reside on specific machines in
> cluster. During operation, regionservers reads and writes region's data, and
> after some time, all this data are reside on local machine, so, all reads
> become local, which is great for performance. Balancer breaks this locality
> during opertation by moving blocks around.
> This patch adds [-exclude <path>] switch, and, if path is provided,
> balancer will not move blocks under this path during operation.
> Attached patch have tested for 0.90.2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
