[ 
https://issues.apache.org/jira/browse/YARN-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429882#comment-16429882
 ] 

Xuan Gong commented on YARN-5193:
---------------------------------

Using the new log format can solve this issue. 

> For long running services, aggregate logs when a container completes instead 
> of when the app completes
> ------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-5193
>                 URL: https://issues.apache.org/jira/browse/YARN-5193
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Priority: Major
>
> For a long running service, containers will typically not complete very 
> often. However, when a container completes - it would be useful to aggregate 
> the logs right then, instead of waiting for the app to complete.
> This will allow the command line log tool to lookup containers for an app 
> from the log file index itself, instead of having to go and talk to YARN. 
> Talking to YARN really only works if ATS is enabled, and YARN is configured 
> to publish container information to ATS (That may not always be the case - 
> since this can overload ATS quite fast).
> There's some added benefits like cleaning out local disk space early, instead 
> of waiting till the app completes. (There's probably a separate jira 
> somewhere about cleanup of container for long running services anyway)
> cc [~vinodkv], [~xgong]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to