Hi Jyotsna, What is your 'decodeErrorPolicy' setting on the spool dir source? By default it is FAIL, but it must be IGNORE or REPLACE in order to proceed further when decoding error occurs.
Regrads, Peter Turcsanyi On Thu, Jun 21, 2018 at 2:54 PM, Jyotsna G <[email protected]> wrote: > Hi All, > Am currently using Flume 1.7.0. > > While processing a set of files, am encountering the below exception and > Flume hangs unpredictably. > > 18/06/21 05:40:40 ERROR source.SpoolDirectorySource: FATAL: Spool > Directory source src-tpa_idoru: { spoolDir: /tmp/tpa-idoru/ }: Uncaught > exception in SpoolDirectorySource thread. Restart or reconfigure Flume to > continue processing. > java.nio.charset.MalformedInputException: Input length = 1 > at java.nio.charset.CoderResult.throwException(CoderResult.java:281) > at org.apache.flume.serialization.ResettableFileInputStream.readChar( > ResettableFileInputStream.java:283) > at org.apache.flume.serialization.LineDeserializer.readLine( > LineDeserializer.java:132) > at org.apache.flume.serialization.LineDeserializer.readEvent( > LineDeserializer.java:70) > at org.apache.flume.serialization.LineDeserializer.readEvents( > LineDeserializer.java:89) > at org.apache.flume.client.avro.ReliableSpoolingFileEventReade > r.readDeserializerEvents(ReliableSpoolingFileEventReader.java:343) > at org.apache.flume.client.avro.ReliableSpoolingFileEventReade > r.readEvents(ReliableSpoolingFileEventReader.java:331) > at org.apache.flume.source.SpoolDirectorySource$ > SpoolDirectoryRunnable.run(SpoolDirectorySource.java:250) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > > This issue looks similar to https://jira.apache.org/jira/browse/FLUME-2052 > > Wouldn't this fix be already included in 1.7? How do I enable flume to > skip these errors and proceed further? > > Thanks, > Jyotsna >
