Sangjin Lee commented on YARN-4238:

Sorry it took me a while to get to this.

The changes look good mostly. It would be good to address the checkstyle issues 
and others as much as we can as they are basically the tech debt we have been 
accumulating. Thanks!

[~varun_saxena] said:
Now coming to what if client does not send it. This would be an issue if 
entities have to be returned sorted by created time or filtering on the basis 
of created time range has to be done. This can be explicitly stated for clients 
that if you do not report created time then we cannot guarantee order while 
fetching multiple entities. This will make it simple from an implementation 
viewpoint. If not, maybe we can cache it and check if entity has gone in to the 
backed or not and based on that, set created time.
But in this case, issue is what if daemon(having the writer) goes down. Maybe 
we can store this info in a state store. But do we need to do that ?

I think it is reasonable to say that the clients are required to set creation 
time and modification time, or they will not be present in the data and things 
like sort will not work correctly on those records. What do you think?

If we do want to handle the case of missing creation time or modification time 
(that's an if), I think a co-processor might be the only reliable option for 
this (I spoke with [~vrushalic] on this). We do want to avoid having to read 
back data to do it. Some use cases for the co-processor:
- filling in missing creation time stamp if it is missing
- setting the modification timestamp only if the new value is later than the 
old value
- setting the creation timestamp only if it the new value is older than the old 

But running co-processors would add overhead and maintenance issues, so it 
would be good if we can avoid it... I'd like to hear what others think.

> createdTime and modifiedTime is not reported while publishing entities to 
> ATSv2
> -------------------------------------------------------------------------------
>                 Key: YARN-4238
>                 URL: https://issues.apache.org/jira/browse/YARN-4238
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4238-YARN-2928.01.patch, 
> YARN-4238-feature-YARN-2928.02.patch
> While publishing entities from RM and elsewhere we are not sending created 
> time. For instance, created time in TimelineServiceV2Publisher class and for 
> other entities in other such similar classes is not updated. We can easily 
> update created time when sending application created event. Likewise for 
> modification time on every write.

This message was sent by Atlassian JIRA

Reply via email to