[
https://issues.apache.org/jira/browse/MAPREDUCE-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated MAPREDUCE-1648:
-------------------------------------
Status: Open (was: Patch Available)
Sorry to come in late, the patch has gone stale. Can you please rebase? Thanks.
Given this is not an issue with MRv2 should we still commit this? I'm happy to,
but not sure it's useful. Thanks.
> Use rolling to limit tasklogs
> -----------------------------
>
> Key: MAPREDUCE-1648
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1648
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Reporter: Guilin Sun
> Labels: logging
> Attachments: MAPREDUCE-1648-v4.patch, mapreduce-1648-design.pdf,
> syslog-baidu-v2.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> There are at least two types of task-logs: syslog and stdlog
> Task-Jvm outputs syslog by log4j with TaskLogAppender, TaskLogAppender looks
> just like "tail -c", it stores last N byte/line logs in memory(via queue),
> and do real output only if all logs is commit and Appender is going to close.
> The common problem of TaskLogAppender and 'tail -c' is keep everything in
> memory and user can't see any log output while task is in progress.
> So I'm going to try RollingFileAppender instead of TaskLogAppender, use
> MaxFileSize&MaxBackupIndex to limit log file size.
> RollingFileAppender is also suitable for stdout/stderr, just redirect
> stdout/stderr to log4j via LoggingOutputStream, no client code have to be
> changed, and RollingFileAppender seems better than 'tail -c' too.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
