[
https://issues.apache.org/jira/browse/HDFS-12993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327487#comment-16327487
]
Ajay Kumar commented on HDFS-12993:
-----------------------------------
{quote}{{getBlocks()}} is more expensive when there are many smaller blocks.
E.g. more blocks to go through to gather 10GB worth. Doing path exclusion in
such clusters can have a huge performance impact.
{quote}
[~kihwal], Seems blocks smaller than \{{dfs.balancer.getBlocks.min-block-size}}
are excluded before this new path exclusion check happens. However i think we
can safely update check for minBlockSize to \{{if (curBlock.getNumBytes() <=
getBlocksMinBlockSize)}}.
[~brahmareddy], Some comments:
# {\{Balancer#excludedPaths}} doesn't set excludedPaths to
\{{BalancerParameters.Builder}}\{code}else if
("-excludedPaths".equalsIgnoreCase(args[i])) {
checkArgument(++i < args.length,
"exclude paths is missing: args = " + Arrays.toString
(args));
excludedPaths = args[i];
}\{code}
# Checkstyle issues in BalancerParameters(L109) ,Dispatcher (L107)
# Could you please add Javadoc for {{Dispatcher#getFilteredPaths,
BlockManager#excludeBlocks}}
# Can we move the check for excludePathPrefixes.isEmpty or null before we call
\{{BlockManager#excludeBlocks}}
> Add an option to Balancer for excluding the paths
> -------------------------------------------------
>
> Key: HDFS-12993
> URL: https://issues.apache.org/jira/browse/HDFS-12993
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover
> Reporter: Brahma Reddy Battula
> Assignee: Brahma Reddy Battula
> Priority: Major
> Attachments: HDFS-12993.patch
>
>
> *Usecase:* Users/Customers want to keep the important/frequently used
> blocks(where blocks are not written with favoured nodes and block pinning is
> not enabled) when balancer is triggered.This can be useful datacentric jobs
> which will run frequently.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
