[
https://issues.apache.org/jira/browse/HADOOP-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HADOOP-6858:
----------------------------
Status: Open (was: Patch Available)
Hi Andrew,
Can you rebase your patch for trunk please? Also, when rotating the logs over,
perhaps an additional option, if set, can also help remove older logs
(max-backup-index-like), thereby supporting both log roll and retention?
> Enable rotateable JVM garbage collection logs for Hadoop daemons
> ----------------------------------------------------------------
>
> Key: HADOOP-6858
> URL: https://issues.apache.org/jira/browse/HADOOP-6858
> Project: Hadoop Common
> Issue Type: New Feature
> Components: scripts
> Affects Versions: 0.22.0
> Reporter: Andrew Ryan
> Attachments: HADOOP-6858.patch
>
>
> The purpose of this enhancement is to make it easier to collect garbage
> collection logs and insure that they persist across restarts in the same way
> that the standard output files of Hadoop daemon JVM's currently does.
> Garbage collection logs are a vital debugging tool for administrators and
> developers. In our production environments, at some point or another, every
> single type of Hadoop daemon has OOM'ed or experienced other significant
> issues related to GC and/or lack of heap memory. For the longest time, we
> have put in garbage collection logs in our HADOOP_NAMENODE_OPTS,
> HADOOP_JOBTRACKER_OPTS, etc. by using options like "-XX:+PrintGCDateStamps
> -XX:+PrintGCDetails -Xloggc:$HADOOP_LOG_DIR/jobtracker.gc.log".
> Unfortunately, these logs don't survive a restart of the node, so if a node
> OOM's and then is restarted automatically, or manually by someone who is
> unaware, we lose the GC logs forever. We also have to manually add GC log
> options to each daemon. This patch:
> 1) Creates a single, optional, off by default, parameter for specifying GC
> logging.
> 2) If that parameter is set, automatically enables GC logging for all daemons
> in the cluster. The parameter is flexible enough to allow for the different
> ways various vendor's JVM's require garbage collection logging to be
> specified.
> 3) If GC logging is on, insures that the GC log files for each daemon are
> rotated with up to 5 copies kept, same as the .out files currently.
> We are currently running a variation of this patch in our 0.20 install. This
> patch actually includes changes to common, mapred, and hdfs, so it obviously
> cannot be applied as-is, but is included here for review and comments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
