[
https://issues.apache.org/jira/browse/YARN-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15304555#comment-15304555
]
Li Lu edited comment on YARN-5169 at 5/27/16 6:51 PM:
------------------------------------------------------
Thanks [~sjlee0] for raising this issue! I noticed that in RM's
SystemMetricsPublisher code, we're creating {{SystemMetricEvent}} s which will
always have timestamps. Those timestamps are typically generated by the
call-sites of each system metrics publisher call. For example, in
{{appACLsUpdated}} call, the update time is generated in
RMAppManager#createAndPopulateNewRMApp. Similarly, appFinished method of SMP is
called in RMAppImpl, when we perform the final state transition to final state.
We also generate the finish time at that time. Maybe we'd like to provide a
similar mechanism in the NM? That is to say, we can have NMMetricsEvent exposed
in NM's source code, and create those events when we publish the data (instead
of using timestamps of existing events)? I think we may put part of this work
in trunk, but one concern is the newly introduced NMMetricsEvent will not be
used in trunk but just in YARN-2928 branch...
was (Author: gtcarrera9):
Thanks [~sjlee0] for raising this issue! I noticed that in RM's
SystemMetricsPublisher code, we're creating {{SystemMetricEvent}}s which will
always have timestamps. Those timestamps are typically generated by the
call-sites of each system metrics publisher call. For example, in
{{appACLsUpdated}} call, the update time is generated in
RMAppManager#createAndPopulateNewRMApp. Similarly, appFinished method of SMP is
called in RMAppImpl, when we perform the final state transition to final state.
We also generate the finish time at that time. Maybe we'd like to provide a
similar mechanism in the NM? That is to say, we can have NMMetricsEvent exposed
in NM's source code, and create those events when we publish the data (instead
of using timestamps of existing events)? I think we may put part of this work
in trunk, but one concern is the newly introduced NMMetricsEvent will not be
used in trunk but just in YARN-2928 branch...
> most YARN events have timestamp of -1
> -------------------------------------
>
> Key: YARN-5169
> URL: https://issues.apache.org/jira/browse/YARN-5169
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Affects Versions: 2.7.2
> Reporter: Sangjin Lee
>
> Most of the YARN events (subclasses of {{AbstractEvent}}) have timestamp of
> -1. {{AbstractEvent}} have two constructors, one that initializes the
> timestamp to -1 and the other to the caller-provided value. But most events
> use the former (thus timestamp of -1).
> Some of the more common events, including {{ApplicationEvent}},
> {{ContainerEvent}}, {{JobEvent}}, etc. do not set the timestamp.
> The rationale for this behavior seems to be mentioned in {{AbstractEvent}}:
> {code}
> // use this if you DON'T care about the timestamp
> public AbstractEvent(TYPE type) {
> this.type = type;
> // We're not generating a real timestamp here. It's too expensive.
> timestamp = -1L;
> }
> {code}
> This absence of the timestamp isn't really visible in many cases and
> therefore may have gone unnoticed, but the timeline service exposes this
> problem very visibly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
