Siddharth Seth created YARN-5193:
------------------------------------
Summary: For long running services, aggregate logs when a
container completes instead of when the app completes
Key: YARN-5193
URL: https://issues.apache.org/jira/browse/YARN-5193
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Siddharth Seth
For a long running service, containers will typically not complete very often.
However, when a container completes - it would be useful to aggregate the logs
right then, instead of waiting for the app to complete.
This will allow the command line log tool to lookup containers for an app from
the log file index itself, instead of having to go and talk to YARN. Talking to
YARN really only works if ATS is enabled, and YARN is configured to publish
container information to ATS (That may not always be the case - since this can
overload ATS quite fast).
There's some added benefits like cleaning out local disk space early, instead
of waiting till the app completes. (There's probably a separate jira somewhere
about cleanup of container for long running services anyway)
cc [~vinodkv], [~xgong]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
