Hi all,
I've been experimenting with using Flume to transport Flume's own logs.
My setup is that I have the apache-log4j-extras RollingFileAppender rolling
the file every minute into a spool directory. Then I have Flume with the
spooldir source reading that directory.
Very often I see:
java.lang.IllegalStateException: File name has been re-used with different
files. Spooling assumpti
on violated for
/opt/castellan/log/spool/castellan-reader.20130430T2055.log.COMPLETED
at
org.apache.flume.client.avro.SpoolingFileLineReader.retireCurrentFile(SpoolingFileLineRe
ader.java:272)
Then it never recovers, repeatedly saying:
2013-05-08 15:57:19,233 ERROR [pool-10-thread-1]
org.apache.flume.source.SpoolDirectorySource Uncau
ght exception in Runnable
java.io.IOException: Stream closed
at java.io.BufferedReader.ensureOpen(BufferedReader.java:115)
at java.io.BufferedReader.readLine(BufferedReader.java:310)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at
org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:180)
I note that 1.4 has extensive changes to the spooldir source but nothing
that really seemed to speak to this.
Does anybody else see this? Anything I should look for? Or is this some
kind of race condition between log4j and spooldir?
Cheers,
Edward
