I use the flume-daemon.sh script to initialize my agents. In that script, I define the FLUME_LOG_DIR and FLUME_LOGFILE environment variables so that my log file ends up being in: /var/log/flume/flume-flume-node-ip-10-114-81-9.log.
Daily, a new log file with the day prior's date extension is created (e.g, flume-flume-node-ip-10-114-81-9.log.2012-03-06). The original log file without the date extension is overwritten. And both files then have a first entry with a time stamp of the new day (e.g., 2012-03-07 00:00:09,084). Then, Flume begins log to both of these files concurrently. Running diff on them shows that they're different in many places, but not entirely so. I don't have any log rotation set up on the /var/log/flume directory, so it must be an automatic Flume setting to rotate the file daily. This started happening only after upgrading to 094-cdh3u2. I'm also seeing it happen on my collector servers, whereas on my master (which I initialize via a shell script with this command: /usr/local/flume/bin/flume master >> /var/log/flume/master_collector.log), nothing has been written to the file for several days and it has a 0 byte size--though the master seems to be running okay. Is anyone else experiencing this problem? Am I missing something about log rotation where this could be happening outside of Flume? Where would the setting to control this be, if Flume itself is responsible? Thanks, JS
