Zhijie Shen commented on YARN-2517:

Thanks for sharing your idea, Hitesh, Mit and Sangjin! I'd like to make some 
clarification. The error that the handler wants to take care of is not the 
communication problem, but the problem that happens when the server is 
processing the posted timeline entity (See TimelinePutResponse). It could be 
the data integrity issue that is messed up by the app. For example, the posted 
Entity A in Domain 1 is trying to relate to Entity B in Domain 2. It's fine if 
in some use cases, the app doesn't want to care about it, and consequently 
doesn't require an Ack. The app can just go ahead without providing the 
handler. However, it's still better to be generic enough to cover the other use 
cases where the app wants to make sure it has the timeline data persisted, or 
at least know it is succeeded or not.

Queueing/messaging layer may help to mitigate the communication problem, but 
aforementioned data problem still could happen, such that the app may still 
want to hear about the put response. However, it sounds right that the 
implementation of this layer will affect that of async call. It makes sense to 
wait until we make it clear how to make client <-> TS communication reliable. 
If we eventually find handler in async call is difficult and will further 
prevent optimization, "a sync write (as it stands now) for critical data and an 
async write which is basically fire-and-forge" sounds a reasonable alternative 

> Implement TimelineClientAsync
> -----------------------------
>                 Key: YARN-2517
>                 URL: https://issues.apache.org/jira/browse/YARN-2517
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-2517.1.patch, YARN-2517.2.patch
> In some scenarios, we'd like to put timeline entities in another thread no to 
> block the current one.
> It's good to have a TimelineClientAsync like AMRMClientAsync and 
> NMClientAsync. It can buffer entities, put them in a separate thread, and 
> have callback to handle the responses.

This message was sent by Atlassian JIRA

Reply via email to