Xianyin Xin commented on YARN-4177:

Hi [~ste...@apache.org], thanks for your comment. I've read your post and did 
some investgations on this.
1.Inconsistent across cores, hence non-monotonic on reads, especially reads 
likely to trigger thread suspend/resume (anything with sleep(), wait(), IO, 
accessing synchronized data under load).
This was once a bug on some old OSs, but it seems not a problem on Linux newer 
than 2.6 or windows newer than XP SP2, if i understand your comment correctly. 
 and the refered 
2.Not actually monotonic.
Can you explain in detail? As a reference, there're some discussion on 
clock_gettime which nanoTime depends in 
 especially in the second post that has 4 supports.
3.Achieving a consistency by querying heavyweight counters with possible longer 
function execution time and lower granularity than the wall clock.
That is: modern NUMA, multi-socket servers are essentially multiple computers 
wired together, and we have a term for that: distributed system
You mean achieving a consistent time across nodes in a cluster? I think the 
monotonic time we plan to offer should be limited to node-local. It's hard to 
make it cluster wide. 
I've known for a long time that CPU frequency could change its rate
I remembered Linux higher than 2.6.18 takes some measures to overcome this 
 has little discussion.

> yarn.util.Clock should not be used to time a duration or time interval
> ----------------------------------------------------------------------
>                 Key: YARN-4177
>                 URL: https://issues.apache.org/jira/browse/YARN-4177
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Xianyin Xin
>            Assignee: Xianyin Xin
>         Attachments: YARN-4177.001.patch, YARN-4177.002.patch
> There're many places uses Clock to time intervals, which is dangerous as 
> commented by [~ste...@apache.org] in HADOOP-12409. Instead, we should use 
> hadoop.util.Timer#monotonicNow() to get monotonic time. Or we could provide a 
> MonotonicClock in yarn.util considering the consistency of code.

This message was sent by Atlassian JIRA

Reply via email to