[
https://issues.apache.org/jira/browse/MAPREDUCE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Travis Crawford updated MAPREDUCE-2589:
---------------------------------------
Attachment: cleanup_userlogs.py
We see this on our clusters too.
Attached is a script that I run from cron to cleanup old userlogs. The general
idea is setting some high water mark for userlog disk space, and when passed,
delete logs until passing some low water mark. Logs for running jobs are
excluded from cleanup, which has infrequently caused issues but in general are
worth excluding.
Posting as an example of what the replacement might look like (as an internal
periodic task, of course). Also, not sure how the nextgen stuff deals with
cleanup.
> TaskTracker not purging userlog directories
> -------------------------------------------
>
> Key: MAPREDUCE-2589
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2589
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.205.0
> Environment: 0.20.205
> Reporter: Sherry Chen
> Assignee: Sherry Chen
> Priority: Minor
> Attachments: cleanup_userlogs.py
>
>
> UserLogCleaner is not robust. Leftover userlogs after a restart sometimes
> have to be manually
> cleaned. Things can accumulate over a period of time.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
