First a bit of context, I'm using logrotate to monitor and copy (cp -p) log files to a flume spooling directory source. So every hour, logrotate checks for and copies a file from the source to the flume destination. I see the following warning message in the flume logs.
17 Nov 2012 14:47:07,682 WARN [pool-10-thread-1] (org.apache.flume.client.avro.SpoolingFileLineReader.getNextFile:328) - Could not find file: /mnt/flume/clickstream/clickstream.log-2012-11-17-1353163623 java.io.FileNotFoundException: /mnt/flume/clickstream/clickstream.log-2012-11-17-1353163623 (Permission denied) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:138) at java.io.FileReader.<init>(FileReader.java:72) at org.apache.flume.client.avro.SpoolingFileLineReader.getNextFile(SpoolingFileLineReader.java:322) at org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:172) at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:135) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Although it appears that Flume processes the log, I'm curious why I''m seeing this and if I have anything with permissions incorrect? Here's the permissions: source log directory under /var/log: drwxrwxr-x 2 ubuntu ubuntu 4096 Nov 17 14:47 clickstream source files: -rw-rw-r-- 1 ubuntu ubuntu 9055750 Nov 17 13:29 clickstream.log-2012-11-17-1353158953.gz -rw-rw-r-- 1 ubuntu ubuntu 13583565 Nov 17 14:17 clickstream.log-2012-11-17-1353161821.gz -rw-rw-r-- 1 ubuntu ubuntu 131296672 Nov 17 14:47 clickstream.log-2012-11-17-1353163623 -rw-rw-r-- 1 ubuntu ubuntu 65648336 Nov 17 14:52 clickstream.log flume source directory under /mnt/flume: drwxrwxr-x 2 ubuntu ubuntu 4096 Nov 17 14:48 clickstream flume source files: -rw-rw-r-- 1 ubuntu ubuntu 131296672 Nov 17 13:29 clickstream.log-2012-11-17-1353158953.COMPLETED -rw-rw-r-- 1 ubuntu ubuntu 196945008 Nov 17 14:17 clickstream.log-2012-11-17-1353161821.COMPLETED -rw-rw-r-- 1 ubuntu ubuntu 131296672 Nov 17 14:47 clickstream.log-2012-11-17-1353163623.COMPLETED Any insight would be appreciated. Regards, Dan
