Sangjin Lee commented on YARN-3045:

Sorry was not able to completely get it. Are you referring to TimelineClient's 
putEntities & putEntitiesAsync ? or 
TimelineWriter's write & flush ? if its later, apart from RM most of the 
entities and events of NM and application go through TimelineClient > 
TimelineCollector . So there should be some way for NM and Client App(AM) to 
inform to flush the data right?

Sorry if my comments were not detailed enough. As you know, we have layers of 
APIs: {{TimelineClient}} which clients (RM, NM, AM, ...) use, 
{{TimelineCollector}} that receives those calls and interacts with the writer, 
and {{TimelineWriter}} that handles actual writes.

The point was that {{flush()}} belongs only in {{TimelineWriter}}. Along with 
that, it is implicitly understood that {{TimelineWriter.write()}} may be 
asynchronous (i.e. may not write the data to the storage synchronously or 
promptly). This API should be sufficient for {{TimelineCollector}} to express 
synchronous/critical put operations and asynchronous/non-critical put 
operations. {{TimelineCollector}} will *not* expose {{flush()}} directly to 
clients. Instead, it may use things like {{putEntity()}} and 
{{putEntityAsync()}} to expose that semantics to the client. In the simplest 
terms, {{TimelineCollector}} could implement {{putEntity()}} as 
{{putEntityAsync()}} + {{TimelineWriter.flush()}}. This is not the actual 
suggestion of the implementation, but that would be an idea. We already have 
{{putEntity()}} and {{putEntityAsync()}}, but we haven't yet used {{flush()}} 
to do this behavior.

And yes, we'll need another JIRA to differentiate {{putEntity()}} and 
{{putEntityAsync()}} and use them at the right places. Currently, 
{{putEntity()}} does not call {{flush()}}, and all client calls are using 

But some of them related to localization i feel is related to ContainerEntity 
right? hope the approach captured by me is fine ?

If these events are already associated with containers any way, they are not an 
issue, right? I thought there were these events that are really application 
events, but specific to the nodes? My comments were about those application 

> [Event producers] Implement NM writing container lifecycle events to ATS
> ------------------------------------------------------------------------
>                 Key: YARN-3045
>                 URL: https://issues.apache.org/jira/browse/YARN-3045
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3045-YARN-2928.002.patch, 
> YARN-3045-YARN-2928.003.patch, YARN-3045-YARN-2928.004.patch, 
> YARN-3045-YARN-2928.005.patch, YARN-3045-YARN-2928.006.patch, 
> YARN-3045.20150420-1.patch
> Per design in YARN-2928, implement NM writing container lifecycle events and 
> container system metrics to ATS.

This message was sent by Atlassian JIRA

Reply via email to