Gera Shegalov created MAPREDUCE-5773:
----------------------------------------
Summary: Provide dedicated MRAppMaster syslog length limit
Key: MAPREDUCE-5773
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5773
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mr-am
Affects Versions: 2.3.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Priority: Blocker
Fix For: 2.4.0
Currently mapreduce.task.userlog.limit.kb controls both the length of task
attempt logs and MA-AM attempt logs. Obviously, MR-AM log is not userlog.
We are interested in keeping MR-AM log either in its entirety or in much larger
size than task logs for debugging.
MAPREDUCE-5672 introduced CRLA. We already use a dedicated setting for how many
backups of rolled files we keep for MR-AM. However, for large jobs with tens of
megabyte AM log, it means that you have a long series of files. A natural
improvement is to have a configuration
yarn.app.mapreduce.am.container.log.limit.kb . It allows to either disable
rolling the log for altogether while keeping it for the task attempts, or to
use much a larger limit to make rolling less frequent.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
