Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17997
Change subject: IMPALA-5256: Force log rolling when max_log_size exceeded ...................................................................... IMPALA-5256: Force log rolling when max_log_size exceeded Impala backends allow STDOUT/STDERR redirection into INFO/ERROR log respectively though redirect_stdout_stderr startup flag. It does so by redirecting STDOUT/STDERR stream to write into the log file symlink created by glog. There are two problems with this approach: 1. When glog rolls the log and starts writing to a new log file, it updates the symlink. But Impala is not aware that the file pointed by the symlink has changed. So cout/cerr write still goes to the oldest log file. 2. When there is a lot of write activity to cout/cerr, the log file can grow big. However, glog is not aware of STDOUT/STDERR activity. It only counts the message bytes that were written to glog (LOG(INFO), LOG(ERROR)). Thus, it only uses its internal bytes count when deciding to roll the logs. This commit addresses the issue by monitoring the log file size every second. If Impala sees that the log file has exceeded max_log_size, it will call google::FlushLogFiles(), ahead of logbufsecs. If the log file stays big after the flush, we will force glog to roll the log. Since there is no direct way to force glog to roll, we do this by changing the log extension to random extension through google::SetLogFilenameExtension(), and immediately return them to extensionless (empty string extension). We also check periodically whether the log file symlink has pointed to a new file. If it has changed, we reattach the STDOUT/STDERR to the new log file. Testing: - Pass the core test. - Add custom cluster test TestLogging::test_excessive_cerr. Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 --- M be/src/common/init.cc M be/src/common/logging.cc M be/src/common/logging.h M be/src/util/filesystem-util.cc M be/src/util/filesystem-util.h M tests/custom_cluster/test_breakpad.py 6 files changed, 198 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/17997/1 -- To view, visit http://gerrit.cloudera.org:8080/17997 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I1b94727180354fe69989ebf3cd1a8f8cda1cf0c3 Gerrit-Change-Number: 17997 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto <[email protected]>
