Thank you so much Peter. I set 'decodeErrorPolicy' to IGNORE and its processing fine now.
Regards, Jyotsna On Thu, Jun 21, 2018 at 7:07 PM, Peter Turcsanyi <[email protected]> wrote: > Hi Jyotsna, > > What is your 'decodeErrorPolicy' setting on the spool dir source? > By default it is FAIL, but it must be IGNORE or REPLACE in order to > proceed further when decoding error occurs. > > Regrads, > Peter Turcsanyi > > On Thu, Jun 21, 2018 at 2:54 PM, Jyotsna G <[email protected]> wrote: > >> Hi All, >> Am currently using Flume 1.7.0. >> >> While processing a set of files, am encountering the below exception and >> Flume hangs unpredictably. >> >> 18/06/21 05:40:40 ERROR source.SpoolDirectorySource: FATAL: Spool >> Directory source src-tpa_idoru: { spoolDir: /tmp/tpa-idoru/ }: Uncaught >> exception in SpoolDirectorySource thread. Restart or reconfigure Flume to >> continue processing. >> java.nio.charset.MalformedInputException: Input length = 1 >> at java.nio.charset.CoderResult.throwException(CoderResult.java:281) >> at org.apache.flume.serialization.ResettableFileInputStream.rea >> dChar(ResettableFileInputStream.java:283) >> at org.apache.flume.serialization.LineDeserializer.readLine(Lin >> eDeserializer.java:132) >> at org.apache.flume.serialization.LineDeserializer.readEvent(Li >> neDeserializer.java:70) >> at org.apache.flume.serialization.LineDeserializer.readEvents(L >> ineDeserializer.java:89) >> at org.apache.flume.client.avro.ReliableSpoolingFileEventReader >> .readDeserializerEvents(ReliableSpoolingFileEventReader.java:343) >> at org.apache.flume.client.avro.ReliableSpoolingFileEventReader >> .readEvents(ReliableSpoolingFileEventReader.java:331) >> at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryR >> unnable.run(SpoolDirectorySource.java:250) >> at java.util.concurrent.Executors$RunnableAdapter.call( >> Executors.java:511) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu >> tureTask.access$301(ScheduledThreadPoolExecutor.java:180) >> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu >> tureTask.run(ScheduledThreadPoolExecutor.java:294) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> >> This issue looks similar to https://jira.apache.org/jir >> a/browse/FLUME-2052 >> >> Wouldn't this fix be already included in 1.7? How do I enable flume to >> skip these errors and proceed further? >> >> Thanks, >> Jyotsna >> > >
