[
https://issues.apache.org/jira/browse/HADOOP-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Travis Crawford updated HADOOP-6746:
------------------------------------
Status: Patch Available (was: Open)
Release Note: Open log files with append (rather than creating new files)
so file truncation works. This allows log rotation tools to operate correctly
without restarting Hadoop processes.
This patch is useful from an operations perspective because Hadoop processes
are typically long-lived and generate large log files, risking full disks which
cause node failure or pages to operations staff to resolve the issue. By
opening files in append mode they can be truncated, thus freeing up disk space
consumed by logs. Using a log rotation tool that (a) copies a log file aside,
(b) truncates "live" file, and (c) compresses rotates file for later analysis
if needed, is a very common task.
Currently log files are not opened in append mode, so truncation simply fills a
file with nothing, and new bytes are written at the existing offset; disk space
is not reclaimed until restarting the process.
> hadoop-daemon.sh does not append to log files
> ---------------------------------------------
>
> Key: HADOOP-6746
> URL: https://issues.apache.org/jira/browse/HADOOP-6746
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Travis Crawford
>
> Looking in hadoop-daemon.sh we see the start command redirects logs to the
> log file by creating a new file. It does not append to the log.
> nohup nice -n $HADOOP_NICENESS "$HADOOP_HOME"/bin/hadoop --config
> $HADOOP_CONF_DIR $command "$@" > "$log" 2>&1 < /dev/null &
> I believe output should be appended to the log file so logrotate copytruncate
> works. Currently a large log file cannot be truncated without restarting the
> running service, which causes issues in production environments.
> The improved line would be exactly the same, except use >> to open the log
> file in append mode.
> nohup nice -n $HADOOP_NICENESS "$HADOOP_HOME"/bin/hadoop --config
> $HADOOP_CONF_DIR $command "$@" >> "$log" 2>&1 < /dev/null &
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
