[
https://issues.apache.org/jira/browse/HADOOP-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peeyush Bishnoi updated HADOOP-5022:
------------------------------------
Attachment: hadoop-5022.txt
Patch is attached for logcondense.py that will optionally delete the JobTracker
logs and also update the logcondense.py documentation . In fact the patch will
delete the complete job directory inside hod-logs in DFS if option "-a" or
"--all" is set to 'true' .
TaskTracker logs gets deleted if option "-a" or "--all" is set to 'false' or
if option is not set . By default option is set to 'false'.
For example:
python logcondense.py -p ~/hadoop-0.17.0/bin/hadoop -d 7 -c ~/hadoop-conf -l
/user -a true
---
> [HOD] logcondense should delete all hod logs for a user, including jobtracker
> logs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-5022
> URL: https://issues.apache.org/jira/browse/HADOOP-5022
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Reporter: Hemanth Yamijala
> Assignee: Peeyush Bishnoi
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: hadoop-5022.txt
>
>
> Currently, logcondense.py does not delete jobtracker logs that it uploads to
> the DFS when the HOD cluster is deallocated. This will result in the hod-logs
> directory to slowly accumulate a whole bunch of jobtracker logs. Particularly
> for users who run a lot of user jobs, this could fill up the namespace.
> Further these directories will cause the logcondense program to keep
> repeatedly looking at these directories stressing out the namenode. So,
> logcondense.py should optionally also delete the jobtracker logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
