[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073760#comment-16073760
]
Weiwei Yang commented on HDFS-12082:
------------------------------------
Proposed a patch to fix this issue. Otherwise if user reconfigures namenode
interval, the value of property {{dfs.block.invalidate.limit}} will be always
overwritten.
The fix simply honors the configuration of {{dfs.block.invalidate.limit}} and
use it for block invalidate limit, and this will not change when heartbeat
interval changes. Reason is, following formula doesn't really work
{code}
// Default heartbeat is 3s, unless heartbeat is set to bigger than 50s,
// it is always 1000
(1) final int blockInvalidateLimit =
Math.max(20*(int)(heartbeatIntervalSeconds),
DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
// We will not reach the default value here, because we always load defaults
// from hdfs-default.xml. If the property is not found in hdfs-site.xml,
// it simply returns the default value from hdfs-default.xml, which is 1000.
// Even blockInvalidateLimit is something else, it doesn't count.
(2) this.blockInvalidateLimit = conf.getInt(
DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_KEY, blockInvalidateLimit);
{code}
so right now there are two cases
# {{dfs.block.invalidate.limit}} not set explicitly, blockInvalidateLimit=1000
# {{dfs.block.invalidate.limit}} is explicitly set,
blockInvalidateLimit=<value_of_the_property>
in this case, why we still need (1) ? I think we can remove it.
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> ---------------------------------------------------------------------------------------------
>
> Key: HDFS-12082
> URL: https://issues.apache.org/jira/browse/HDFS-12082
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-12082.001.patch
>
>
> HDFS-1477 provides an option to reconfigured namenode heartbeat interval
> without restarting the namenode. When the heartbeat interval is reconfigured,
> {{blockInvalidateLimit}} gets recounted
> {code}
> this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
> DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
> {code}
> this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
