[
https://issues.apache.org/jira/browse/FLUME-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743515#comment-13743515
]
Ted Malaska commented on FLUME-2160:
------------------------------------
I've read through the code and this is a summary of my first design.
In readEvent(int numEvent) method of ReliableSpoopingFileEventReader a try
catch block
should be around line 234.
currentFile.get().getDeserializer().readEvents(numEvents);
If there is an exception then the following should happen:
1) A error message gets logged with the following information
1.1) The file name and path
1.2) The start event before the readEvent method was called and the number of
events that were requested on being read
1.3) The IO Exception from the method
2) The file should be moved to a fileFailedSuffix
2.1) Reuse the logic in the rollCurrentFile to get code reuse
Question to Flume people: Is there a reason why the spoolDirectorySource uses
suffixes over multiple directories.
> SpoolDirectorySource uncaught exception
> ---------------------------------------
>
> Key: FLUME-2160
> URL: https://issues.apache.org/jira/browse/FLUME-2160
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Environment: Linux 2.6.32-358.14.1.el6.x86_64 #1 SMP Tue Jul 16
> 23:51:20 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Rob
>
> Ive noticed some instabilities with Flume, of particular is this uncaught
> exception. One of the bigger challenges with this type of exception, is that
> the agent is now in a hung state repeating the same exception. The tooling
> for inspecting what file/event is causing the issue are lacking or
> non-existant, and architecturally there should be some functionality
> equivalant to the "dead-letter-queue". Here is the exception Im dealing with
> now:
> 13/08/14 15:37:05 ERROR source.SpoolDirectorySource: Uncaught exception in
> Runnable
> java.lang.IllegalStateException: Serializer has been closed
> at
> org.apache.flume.serialization.LineDeserializer.ensureOpen(LineDeserializer.java:124)
> at
> org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:88)
> at
> org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:221)
> at
> org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:154)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
