[
https://issues.apache.org/jira/browse/HDFS-16219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17414929#comment-17414929
]
Viraj Jasani commented on HDFS-16219:
-------------------------------------
I believe there are 2 ways to deal with this:
# Provide non-empty default values of numMaps and bandwidth in
hdfs-rbf-default i.e dfs.federation.router.federation.rename.map value: 20 and
dfs.federation.router.federation.rename.bandwidth value: 100. Default values
are derived from DistCpConstants.
# If we keep hdfs-rbf-default as is, we should update logic in
buildRouterRenameJob to exclude setters setMapNum() and setBandwidthLimit() in
the absence of config values. However, if we go this route, we will also have
to update OptionsParser to derive default values for bandwidth.
{code:java}
/* Default number of maps to use for DistCp */
public static final int DEFAULT_MAPS = 20;
/* Default bandwidth if none specified */
public static final float DEFAULT_BANDWIDTH_MB = 100;
{code}
I think option#1 is simple and better rather than going with option#2 requiring
changes in DistCp module.
> RBF: Set default map tasks and bandwidth in RouterFederationRename
> ------------------------------------------------------------------
>
> Key: HDFS-16219
> URL: https://issues.apache.org/jira/browse/HDFS-16219
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
> Environment: Hadoop 3.3.0 with patches
> Reporter: Akira Ajisaka
> Assignee: Viraj Jasani
> Priority: Major
>
> If dfs.federation.router.federation.rename.map or
> dfs.federation.router.federation.rename.bandwidth is not set, DFSRouter fails
> to launch.
> This issue is similar to HDFS-16217.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
