[
https://issues.apache.org/jira/browse/NIFI-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735934#comment-14735934
]
Mark Payne commented on NIFI-938:
---------------------------------
Hey Brian,
So far I've been unable to replicate this issue. Can you provide a few details
to help me replicate?
Are you seeing any other errors in the logs?
What is the average size of the Kafka messages?
What delimiter are you using in GetKafka to separate the messages?
Do you expect to have any 0-byte messages on the Kafka topic?
Any other interesting configuration options set on GetKafka?
Do you have a sense for whether or not you're caught up with the messages in
Kafka? I.e., is Kafka out of messages, or does it have more that you've not
pulled?
Is the content repo storing data on a local disk or a network drive (EC2 EBS,
for example?)
Looking into this, I did run into an issue with how StandardProcessSession
handled the 'append' method, and I have resolved that locally. However, I am
not sure that it's the same problem. Without the patch I am still seeing
messages coming out properly but the session continually copies the content
over for each message it adds to the FlowFile. I will create a separate ticket
for that and link it to this ticket as related, but I think these could be two
separate issues.
Thanks
-Mark
> Appending to FlowFiles Errors Finding Content - GetKafka
> --------------------------------------------------------
>
> Key: NIFI-938
> URL: https://issues.apache.org/jira/browse/NIFI-938
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 0.3.0
> Reporter: Brian Ghigiarelli
> Priority: Critical
> Fix For: 0.3.0
>
>
> With the latest 0.3.0-SNAPSHOT on the master branch, GetKafka throws errors
> only when the Batch Size is greater than 1. NiFi is unable to find the
> existing FlowFile in order to append new content to it.
> {noformat}
> 2015-09-08 15:36:36,643 ERROR [Timer-Driven Process Thread-3]
> o.apache.nifi.processors.kafka.GetKafka
> GetKafka[id=6eec6998-c271-4de3-b092-42469d9b494c] Failed to receive FlowFile
> from Kafka due to
> org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find
> content for FlowFile:
> org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find
> content for FlowFile
> 2015-09-08 15:36:36,644 ERROR [Timer-Driven Process Thread-3]
> o.apache.nifi.processors.kafka.GetKafka
> org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find
> content for FlowFile
> at
> org.apache.nifi.controller.repository.StandardProcessSession.handleContentNotFound(StandardProcessSession.java:2333)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.append(StandardProcessSession.java:2028)
> ~[na:na]
> at
> org.apache.nifi.processors.kafka.GetKafka.onTrigger(GetKafka.java:313)
> ~[nifi-kafka-processors-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> Caused by: org.apache.nifi.controller.repository.ContentNotFoundException:
> Could not find content for
> org.apache.nifi.controller.repository.claim.StandardContentClaim@90a96e21
> at
> org.apache.nifi.controller.repository.FileSystemRepository.getPath(FileSystemRepository.java:490)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.FileSystemRepository.read(FileSystemRepository.java:795)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.append(StandardProcessSession.java:1988)
> ~[na:na]
> ... 13 common frames omitted
> 2015-09-08 15:36:36,644 ERROR [Timer-Driven Process Thread-3]
> o.apache.nifi.processors.kafka.GetKafka
> GetKafka[id=6eec6998-c271-4de3-b092-42469d9b494c]
> GetKafka[id=6eec6998-c271-4de3-b092-42469d9b494c] failed to process due to
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=5ae54a81-1db5-4106-b4f4-dce41771a6b4,claim=org.apache.nifi.controller.repository.claim.StandardContentClaim@90a96e21,offset=0,name=56655239492779,size=3644]
> is not known in this session (StandardProcessSession[id=175]); rolling back
> session: org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=5ae54a81-1db5-4106-b4f4-dce41771a6b4,claim=org.apache.nifi.controller.repository.claim.StandardContentClaim@90a96e21,offset=0,name=56655239492779,size=3644]
> is not known in this session (StandardProcessSession[id=175])
> 2015-09-08 15:36:36,644 ERROR [Timer-Driven Process Thread-3]
> o.apache.nifi.processors.kafka.GetKafka
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=5ae54a81-1db5-4106-b4f4-dce41771a6b4,claim=org.apache.nifi.controller.repository.claim.StandardContentClaim@90a96e21,offset=0,name=56655239492779,size=3644]
> is not known in this session (StandardProcessSession[id=175])
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2345)
> ~[nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.remove(StandardProcessSession.java:1567)
> ~[nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.processors.kafka.GetKafka.onTrigger(GetKafka.java:338)
> ~[nifi-kafka-processors-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
> [nifi-framework-core-0.3.0-SNAPSHOT.jar:0.3.0-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
