[ https://issues.apache.org/jira/browse/YARN-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Han updated YARN-1130: --------------------------- Attachment: YARN-1130.patch > Improve the log flushing for tasks when mapred.userlog.limit.kb is set > ---------------------------------------------------------------------- > > Key: YARN-1130 > URL: https://issues.apache.org/jira/browse/YARN-1130 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager > Affects Versions: 2.0.5-alpha > Reporter: Paul Han > Attachments: YARN-1130.patch > > > When userlog limit is set with something like this: > {code} > <property> > <name>mapred.userlog.limit.kb</name> > <value>2048</value> > <description>The maximum size of user-logs of each task in KB. 0 disables the > cap. > </description> > </property> > {code} > the log entry will be truncated randomly for the jobs. > The log size is left between 1.2MB to 1.6MB. > Since the log is already limited, avoid the log truncation is crucial for > user. > The other issue with the current > impl(org.apache.hadoop.yarn.ContainerLogAppender) is that log entries will > not flush to file until the container shutdown and logmanager close all > appenders. If user likes to see the log during task execution, it doesn't > support it. > Will propose a patch to add a flush mechanism and also flush the log when > task is done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira