[
https://issues.apache.org/jira/browse/HADOOP-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peeyush Bishnoi updated HADOOP-5022:
------------------------------------
Attachment: hadoop-5022-2.txt
With this new patch minor change has been done in logcondense.py
"--retain-masters-logs" options 'help ' attribute .
Also testing of the patch has been done.
If options " -r " or "--retain-masters-logs" is set to 'true' then , log of
masters ( JobTracker and Namenode if "--dynamicdfs" is set ) , does not get
removed . But If options " -r " or "--retain-masters-logs" is set to 'false'
then complete job directory inside hod-logs in HDFS gets removed .
---
> [HOD] logcondense should delete all hod logs for a user, including jobtracker
> logs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-5022
> URL: https://issues.apache.org/jira/browse/HADOOP-5022
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/hod
> Reporter: Hemanth Yamijala
> Assignee: Peeyush Bishnoi
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: hadoop-5022-1.txt, hadoop-5022-2.txt, hadoop-5022.txt
>
>
> Currently, logcondense.py does not delete jobtracker logs that it uploads to
> the DFS when the HOD cluster is deallocated. This will result in the hod-logs
> directory to slowly accumulate a whole bunch of jobtracker logs. Particularly
> for users who run a lot of user jobs, this could fill up the namespace.
> Further these directories will cause the logcondense program to keep
> repeatedly looking at these directories stressing out the namenode. So,
> logcondense.py should optionally also delete the jobtracker logs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
