[
https://issues.apache.org/jira/browse/YARN-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223923#comment-14223923
]
Zhijie Shen commented on YARN-2517:
-----------------------------------
Thanks for sharing your idea, Hitesh, Mit and Sangjin! I'd like to make some
clarification. The error that the handler wants to take care of is not the
communication problem, but the problem that happens when the server is
processing the posted timeline entity (See TimelinePutResponse). It could be
the data integrity issue that is messed up by the app. For example, the posted
Entity A in Domain 1 is trying to relate to Entity B in Domain 2. It's fine if
in some use cases, the app doesn't want to care about it, and consequently
doesn't require an Ack. The app can just go ahead without providing the
handler. However, it's still better to be generic enough to cover the other use
cases where the app wants to make sure it has the timeline data persisted, or
at least know it is succeeded or not.
Queueing/messaging layer may help to mitigate the communication problem, but
aforementioned data problem still could happen, such that the app may still
want to hear about the put response. However, it sounds right that the
implementation of this layer will affect that of async call. It makes sense to
wait until we make it clear how to make client <-> TS communication reliable.
If we eventually find handler in async call is difficult and will further
prevent optimization, "a sync write (as it stands now) for critical data and an
async write which is basically fire-and-forge" sounds a reasonable alternative
plan.
> Implement TimelineClientAsync
> -----------------------------
>
> Key: YARN-2517
> URL: https://issues.apache.org/jira/browse/YARN-2517
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Zhijie Shen
> Assignee: Tsuyoshi OZAWA
> Attachments: YARN-2517.1.patch, YARN-2517.2.patch
>
>
> In some scenarios, we'd like to put timeline entities in another thread no to
> block the current one.
> It's good to have a TimelineClientAsync like AMRMClientAsync and
> NMClientAsync. It can buffer entities, put them in a separate thread, and
> have callback to handle the responses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
