Naganarasimha G R commented on YARN-3995:

bq. Instead of spawning multiple threads may be we can have single thread which 
does this activity ?
Yes i wanted to address it as i was trying to point out earlier ??Instead of 
spawning multiple threads may be we can have single thread which does this 

bq. How about creating a long-lived single ScheduledExecutorService and 
schedule removeApplication() with the specified delay?
IIUC the approach you mentioned in the callable we will be sleeping for the 
configured period for a application and then remove it. but if multiple apps at 
the same time finish then initial apps only wait for configured period but 
subsequent apps wait for lil more time than the earlier ones.(app's wait period 
+ other apps wait period in the queue ) thoughts?
Some approaches i can adopt to avoid the above issue are :
* Have the timestamp when *close AM container* was called in the callable, and 
in the callable we can have code to wait only if the elapsed time < configured 
linger time.
* Have a map<appid, timestamp> and a single thread(either executor service/ 
timer task) with lower interval like 500ms and it can check this map and remove 
all the apps whose elapsed time is > configured linger time.
thoughts ?

> Some of the NM events are not getting published due race condition when AM 
> container finishes in NM 
> ----------------------------------------------------------------------------------------------------
>                 Key: YARN-3995
>                 URL: https://issues.apache.org/jira/browse/YARN-3995
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-3995-feature-YARN-2928.v1.001.patch
> As discussed in YARN-3045:  While testing in TestDistributedShell found out 
> that few of the container metrics events were failing as there will be race 
> condition. When the AM container finishes and removes the collector for the 
> app, still there is possibility that all the events published for the app by 
> the current NM and other NM are still in pipeline, 

This message was sent by Atlassian JIRA

Reply via email to