[
https://issues.apache.org/jira/browse/NIFI-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15531475#comment-15531475
]
Bryan Bende commented on NIFI-2841:
-----------------------------------
[~dhicks82] Thanks for reporting this, do you think you could provide an
example schema and data file that produces the problem?
I'm not sure it is happening on any and all Avro files because there a bunch of
unit tests for the processor that all work without encountering this:
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-avro-bundle/nifi-avro-processors/src/test/java/org/apache/nifi/processors/avro/TestSplitAvro.java
I do believe there may be a problem though and would like to track it down.
> SplitAvro Processor is Broken
> -----------------------------
>
> Key: NIFI-2841
> URL: https://issues.apache.org/jira/browse/NIFI-2841
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: David Hicks
> Priority: Critical
>
> This is largely the fault of the Avro DataFileStream reader, but it's making
> the processor unusable. The problem appears to occur when you make the
> following series of calls (which happens because of the splitSize comparison):
> reader.next() -> returns last element
> reader.hasNext() -> returns false
> reader.hasNext() -> returns true
> reader.next() -> EOFException
> This should be reproducible with any and all avro files.
> org.apache.nifi.processor.exception.ProcessException: IOException thrown from
> SplitAvro[id=22e03ca4-0151-4474-92fc-040e1fe12ab9]: java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2013)
> ~[na:na]
> at
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter$1.process(SplitAvro.java:250)
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
> ~[na:na]
> at
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter.split(SplitAvro.java:236)
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.processors.avro.SplitAvro.onTrigger(SplitAvro.java:202)
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
> [nifi-framework-core-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
> [nifi-framework-core-0.7.0.jar:0.7.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_101]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_101]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.io.EOFException: null
> at
> org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
> ~[avro-1.7.7.jar:1.7.7]
> at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
> ~[avro-1.7.7.jar:1.7.7]
> at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:363)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:355)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:157)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
> ~[avro-1.7.7.jar:1.7.7]
> at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
> ~[avro-1.7.7.jar:1.7.7]
> at
> org.apache.nifi.processors.avro.SplitAvro$RecordSplitter$1$1.process(SplitAvro.java:259)
> ~[nifi-avro-processors-0.7.0.jar:0.7.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:1998)
> ~[na:na]
> ... 17 common frames omitted
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
