[
https://issues.apache.org/jira/browse/HDFS-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310564#comment-14310564
]
Benoy Antony commented on HDFS-6133:
------------------------------------
I know that there are different ways to accomplish the desired functionality of
pinning the blocks under a directory to a set of nodes.
I want to suggest yet another approach using storage policies.
This is based on the fact that the set of favored nodes can be considered as a
tier.
Let’s say blocks under /hbase needs to be pinned to a set of nodes.
1. Mark the data.dirs on those set of datanodes as a special storage type , say
HBASE_TIER. [HBASE_TIER]/hadoop01/data
2. Define a storage policy - HBASE_POLICY which insists that all blocks
following the policy, reside in HBASE_TIER
3. Set storage policy of /hbase to HBASE_POLICY
Since the balancer(and mover) considers the storage policy while
moving/balancing, balancer will not move these blocks out of the favored
nodes(HBASE_TIER)
Now if we want to keep other blocks ( non hbase blocks) on these datanodes as
well(may not be a great idea), then we can model it by having multiple storage
types on a single directory like [HBASE_TIER,DISK]/hadoop01/data. This make
sense since one want to have blocks having multiple storage characteristics on
the same node.
This will need a (minor) code modification.
> Make Balancer support exclude specified path
> --------------------------------------------
>
> Key: HDFS-6133
> URL: https://issues.apache.org/jira/browse/HDFS-6133
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover, namenode
> Reporter: zhaoyunjiong
> Assignee: zhaoyunjiong
> Attachments: HDFS-6133-1.patch, HDFS-6133-2.patch, HDFS-6133-3.patch,
> HDFS-6133-4.patch, HDFS-6133-5.patch, HDFS-6133-6.patch, HDFS-6133-7.patch,
> HDFS-6133-8.patch, HDFS-6133.patch
>
>
> Currently, run Balancer will destroying Regionserver's data locality.
> If getBlocks could exclude blocks belongs to files which have specific path
> prefix, like "/hbase", then we can run Balancer without destroying
> Regionserver's data locality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
