[
https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787226#comment-16787226
]
Erik Krogen commented on HDFS-14346:
------------------------------------
Hey [~csun], thanks for reporting.
This actually seems a little tricky to solve. I think it's safe to assume that
there are probably many people out there who still use the deprecated form of
specifying the config, without any units. So if we change the call to
{{conf.getTimeDuration(..., ..., TimeUnit.MILLISECONDS)}}, suddenly their log
roll period will be 1000x less than before!
This situation actually seems applicable to many other cases where
{{getTimeDuration()}} is used. Essentially the default unit also becomes the
smallest granularity, but there's no reason these two should be the same. It
seems to me we need a new {{getTimeDuration()}} like:
{code}
long getTimeDuration(String key, String default, TimeUnit defaultUnit, TimeUnit
returnUnit)
{code}
Where a caller can independently specify the default unit and the return unit.
Then we can do:
{code}
logRollPeriodMs = conf.getTimeDuration(
DFSConfigKeys.DFS_HA_LOGROLL_PERIOD_KEY,
DFSConfigKeys.DFS_HA_LOGROLL_PERIOD_DEFAULT,
TimeUnit.SECONDS, TimeUnit.MILLISECONDS);
{code}
> EditLogTailer loses precision for sub-second edit log tailing and rolling
> interval
> ----------------------------------------------------------------------------------
>
> Key: HDFS-14346
> URL: https://issues.apache.org/jira/browse/HDFS-14346
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Chao Sun
> Assignee: Chao Sun
> Priority: Minor
>
> {{EditLogTailer}} currently uses the following:
> {code}
> logRollPeriodMs = conf.getTimeDuration(
> DFSConfigKeys.DFS_HA_LOGROLL_PERIOD_KEY,
> DFSConfigKeys.DFS_HA_LOGROLL_PERIOD_DEFAULT, TimeUnit.SECONDS) * 1000;
> sleepTimeMs = conf.getTimeDuration(
> DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_KEY,
> DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_DEFAULT, TimeUnit.SECONDS) *
> 1000;
> {code}
> to determine the edit log roll and tail frequency. However, if user specify
> sub-second frequency, such as {{100ms}}, it will lose precision and become
> 0s. This is not ideal for some scenarios such as standby reads (HDFS-12943).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
