Naganarasimha G R commented on YARN-3367:

Thanks [~gtCarrera9], 
For looking into this, There were lot of open questions for this jira from my 
end which i mentioned 
 Hence just started with initial approach path. Mainly to use the refactor and 
reuse the existing {{AsyncDispatcher}} we need to discuss on below specific 
points : 
* 3 Is it important to maintain the order of events which are sent from sync 
and async ? i.e. Is it req to ensure all the async events are also pushed along 
with the current sync event or is it ok to send only the sync ? (current patch 
just ensures async events are in order) .
* 4 Whether its req to merge entities of multiple async calls as they belong to 
same application ?
If concluded then i can analyze further and inform.

> Replace starting a separate thread for post entity with event loop in 
> TimelineClient
> ------------------------------------------------------------------------------------
>                 Key: YARN-3367
>                 URL: https://issues.apache.org/jira/browse/YARN-3367
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Junping Du
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3367.YARN-2928.001.patch
> Since YARN-3039, we add loop in TimelineClient to wait for 
> collectorServiceAddress ready before posting any entity. In consumer of  
> TimelineClient (like AM), we are starting a new thread for each call to get 
> rid of potential deadlock in main thread. This way has at least 3 major 
> defects:
> 1. The consumer need some additional code to wrap a thread before calling 
> putEntities() in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting 
> operation thread get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side, 
> putEntities() only put related entities into a queue of entities and a 
> separated thread handle to deliver entities in queue to collector via REST 
> call.

This message was sent by Atlassian JIRA

Reply via email to