[
https://issues.apache.org/jira/browse/HADOOP-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063270#comment-14063270
]
Zhijie Shen commented on HADOOP-10794:
--------------------------------------
I'm afraid each component may more or less have made the assumption that clock
is synchronized across the cluster. Unfortunately, the problem that made us be
aware of this issue came from MR: an elapsed time that based the the diff of
the timestamps on two hosts became negative. Having a try with a cluster whose
hosts are completely out of sync in terms of clock may give us the idea how bad
the situation is.
According to the prior discussion, it seems to be good to have a mechanism to
check the clocks across a cluster, report unacceptable asynchronization,
doesn't it? On the other hand, it is arguable whether this mechanism should be
part of Hadoop or other cluster monitoring tools.
Anyway, IMHO, it's good to at least let users be aware of the clock
synchronization assumption (via jira, mailing list or etc.), and the risks of
being out of sync, such that they can be more careful about this factor when
setting up a cluster.
> A hadoop cluster needs clock synchronization
> --------------------------------------------
>
> Key: HADOOP-10794
> URL: https://issues.apache.org/jira/browse/HADOOP-10794
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Zhijie Shen
>
> As a distributed system, a hadoop cluster wants the clock on all the
> participating hosts synchronized. Otherwise, some problems might happen. For
> example, in YARN-2251, due to the clock on the host for the task container
> falls behind that on the host of the AM container, the computed elapsed time
> (the diff between the timestamps produced on two hosts) becomes negative.
> In YARN-2251, we tried to mask the negative elapsed time. However, we should
> seek for a decent long term solution, such as providing mechanism to do and
> check clock synchronization.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
