Naganarasimha G R commented on YARN-3367:

Thanks for the feed back [~sjlee0],
bq. If the client is fine without the server response, it clearly implies flush 
is not needed 
Yes i agree with this, but what should be the behavior of Sync calls ? IMO in 
the wake of YARN-4061  (Fault tolerant writer for timeline v2), we need not 
worry abt it either, Thoughts ?
bq. First of all, the timestamp should be set for most of the entities, 
metrics, events, etc., and the server should rely on the timestamps to resolve 
Well i can understand if all the events are received and timestamp is filled at 
the client side we need not worry abt the order but in the case client goes 
down send some events out of order ? like containerFinished event gets 
Published but logAggregation does not succeed. And from Client App side not 
sure how important is the order to be maintained.

bq.   I don't see a whole lot of situations where the timeline client can do 
this easily and unambiguously. 
Well the approach i thought is to block publishing further entities/events 
through sync /async calls till the events in Timeline client queue is cleared. 
But i don't completely see the need for them until its very necessary to 
maintain the order. [~djp] can you please comment on this part as in this jira 
description you have targetted to get the events in order.

> Replace starting a separate thread for post entity with event loop in 
> TimelineClient
> ------------------------------------------------------------------------------------
>                 Key: YARN-3367
>                 URL: https://issues.apache.org/jira/browse/YARN-3367
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Junping Du
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3367.YARN-2928.001.patch
> Since YARN-3039, we add loop in TimelineClient to wait for 
> collectorServiceAddress ready before posting any entity. In consumer of  
> TimelineClient (like AM), we are starting a new thread for each call to get 
> rid of potential deadlock in main thread. This way has at least 3 major 
> defects:
> 1. The consumer need some additional code to wrap a thread before calling 
> putEntities() in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting 
> operation thread get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side, 
> putEntities() only put related entities into a queue of entities and a 
> separated thread handle to deliver entities in queue to collector via REST 
> call.

This message was sent by Atlassian JIRA

Reply via email to