[
https://issues.apache.org/jira/browse/NIFI-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989728#comment-15989728
]
Joseph Witt commented on NIFI-3739:
-----------------------------------
this error
2017-04-28 23:22:07,716 ERROR [Timer-Driven Process Thread-7]
o.a.n.p.k.pubsub.PublishKafkaRecord_0_10
PublishKafkaRecord_0_10[id=b24fcf7f-015b-1000-483b-27e62dcb54dd] Failed to send
all mess
age for
StandardFlowFileRecord[uuid=cb16bf5b-3b92-4d7f-bf07-75c87102f025,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1493435997738-5183, container=default,
section=63],
offset=570310, length=222792],offset=0,name=461002784014590,size=222792] to
Kafka; routing to failure due to
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerEx
ception: null of string in field details of nifi.provenanceEvent: {}
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerException: null of string in field details of
nifi.provenanceEvent
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at
org.apache.nifi.avro.WriteAvroResultWithSchema.write(WriteAvroResultWithSchema.java:76)
at
org.apache.nifi.processors.kafka.pubsub.PublisherLease.publish(PublisherLease.java:109)
at
org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_0_10$1.process(PublishKafkaRecord_0_10.java:341)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2120)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2090)
at
org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_0_10.onTrigger(PublishKafkaRecord_0_10.java:336)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:144)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: null of string in field details of
nifi.provenanceEvent
at
org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:132)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:126)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 18 common frames omitted
Caused by: java.lang.NullPointerException: null
at org.apache.avro.io.Encoder.writeString(Encoder.java:121)
at
org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:254)
at
org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:249)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:115)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
... 21 common frames omitted
Led to this output which suggests incorrect failure handling
2017-04-28 23:22:07,723 ERROR [Timer-Driven Process Thread-4]
o.a.n.p.k.pubsub.PublishKafkaRecord_0_10
PublishKafkaRecord_0_10[id=b24fcf7f-015b-1000-483b-27e62dcb54dd]
PublishKafkaRecord_0_10[id=b24fcf7f-015b-1000-483b-27e62dcb54dd] failed to
process due to org.apache.nifi.processor.exception.FlowFileHandlingException:
StandardFlowFileRecord[uuid=093eb60f-00e5-4535-a4d2-786a50beaeb5,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1493435997738-5183, container=default,
section=63], offset=0,
length=292442],offset=0,name=461002471111728,size=292442] is not the most
recent version of this FlowFile within this session
(StandardProcessSession[id=1417184]); rolling back session: {}
org.apache.nifi.processor.exception.FlowFileHandlingException:
StandardFlowFileRecord[uuid=093eb60f-00e5-4535-a4d2-786a50beaeb5,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1493435997738-5183, container=default,
section=63], offset=0,
length=292442],offset=0,name=461002471111728,size=292442] is not the most
recent version of this FlowFile within this session
(StandardProcessSession[id=1417184])
at
org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2865)
at
org.apache.nifi.controller.repository.StandardProcessSession.transfer(StandardProcessSession.java:1799)
at
org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_0_10.onTrigger(PublishKafkaRecord_0_10.java:382)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1118)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:144)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
> Create Processors for publishing records to and consuming records from Kafka
> ----------------------------------------------------------------------------
>
> Key: NIFI-3739
> URL: https://issues.apache.org/jira/browse/NIFI-3739
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Fix For: 1.2.0
>
>
> With the new record readers & writers that have been added in now, it would
> be good to allow records to be pushed to and pulled from kafka. Currently, we
> support demarcated data but sometimes we can't correctly demarcate data in a
> way that keeps the format valid (json is a good example). We should have
> processors that use the record readers and writers for this.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
