[ https://issues.apache.org/jira/browse/YARN-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223923#comment-14223923 ]
Zhijie Shen commented on YARN-2517: ----------------------------------- Thanks for sharing your idea, Hitesh, Mit and Sangjin! I'd like to make some clarification. The error that the handler wants to take care of is not the communication problem, but the problem that happens when the server is processing the posted timeline entity (See TimelinePutResponse). It could be the data integrity issue that is messed up by the app. For example, the posted Entity A in Domain 1 is trying to relate to Entity B in Domain 2. It's fine if in some use cases, the app doesn't want to care about it, and consequently doesn't require an Ack. The app can just go ahead without providing the handler. However, it's still better to be generic enough to cover the other use cases where the app wants to make sure it has the timeline data persisted, or at least know it is succeeded or not. Queueing/messaging layer may help to mitigate the communication problem, but aforementioned data problem still could happen, such that the app may still want to hear about the put response. However, it sounds right that the implementation of this layer will affect that of async call. It makes sense to wait until we make it clear how to make client <-> TS communication reliable. If we eventually find handler in async call is difficult and will further prevent optimization, "a sync write (as it stands now) for critical data and an async write which is basically fire-and-forge" sounds a reasonable alternative plan. > Implement TimelineClientAsync > ----------------------------- > > Key: YARN-2517 > URL: https://issues.apache.org/jira/browse/YARN-2517 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Zhijie Shen > Assignee: Tsuyoshi OZAWA > Attachments: YARN-2517.1.patch, YARN-2517.2.patch > > > In some scenarios, we'd like to put timeline entities in another thread no to > block the current one. > It's good to have a TimelineClientAsync like AMRMClientAsync and > NMClientAsync. It can buffer entities, put them in a separate thread, and > have callback to handle the responses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)