Matt Burgess created NIFI-4349:
----------------------------------
Summary: AbstractRouteRecord needs to clean up resources on error
Key: NIFI-4349
URL: https://issues.apache.org/jira/browse/NIFI-4349
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Matt Burgess
Reproduction environment:
CSVReader with access strategy set to "Use String Fields from Header"
CSVRecordSetWriter with Schema Write Strategy set to "Write schema.name
attribute" <-- NOTE: This is an incorrect configuration, explained below
IPLookupService with local test MaxMind DB file
GenerateFlowFile -> LookupRecord -> LogAttribute
The IP addresses in the flow file are not present in the DB, but that doesn't
seem to be the issue (I'm just pointing it out because it shows up in the log
below). Instead, when the CSVRecordSetWriter tries to write the schema.name
attribute, it can't because no schema name was ever chosen (the incoming schema
is defined by the CSV header). However, rather than routing the incoming flow
file to failure (which looks like the intent of the code), instead the session
is rolled back and the incoming flow file returned to the queue. This is
because the output flow file had been created but not removed on the following
error:
2017-09-05 13:24:36,776 WARN [Timer-Driven Process Thread-3]
o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.processor.exception.FlowFileHandlingException:
StandardFlowFileRecord[uuid=2d59a235-234b-40f0-9d1d-edb57682e71c,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1504632264821-1, container=default,
section=1], offset=285, length=0],offset=0,name=10043674590562,size=0] transfer
relationship not specified
at
org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:251)
at
org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:321)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2017-09-05 13:24:37,778 WARN [Timer-Driven Process Thread-3]
o.a.nifi.lookup.maxmind.IPLookupService
IPLookupService[id=530bf885-015e-1000-bd99-e47601e7d170] Could not resolve the
IP for value '{ip=ip}'. This is usually caused by issue resolving the
appropriate DNS record or providing the service with an invalid IP address:
2017-09-05 13:24:37,779 ERROR [Timer-Driven Process Thread-3]
o.a.n.processors.standard.LookupRecord
LookupRecord[id=53067120-015e-1000-9cb1-49ff6287d8ce] Failed to process
StandardFlowFileRecord[uuid=6e3b6159-885e-449d-80ab-919cd808941c,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1504632264821-1, container=default,
section=1], offset=0, length=285],offset=0,name=10043674590562,size=285]:
org.apache.nifi.processor.exception.ProcessException: Could not parse incoming
data
org.apache.nifi.processor.exception.ProcessException: Could not parse incoming
data
at
org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:151)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2174)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2144)
at
org.apache.nifi.processors.standard.AbstractRouteRecord.onTrigger(AbstractRouteRecord.java:120)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.nifi.schema.access.SchemaNotFoundException: Cannot write
Schema Name As Attribute because the Schema Name is not known
at
org.apache.nifi.schema.access.SchemaNameAsAttribute.validateSchema(SchemaNameAsAttribute.java:53)
at
org.apache.nifi.serialization.SchemaRegistryRecordSetWriter.getSchemaAccessWriter(SchemaRegistryRecordSetWriter.java:137)
at
org.apache.nifi.csv.CSVRecordSetWriter.createWriter(CSVRecordSetWriter.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89)
at com.sun.proxy.$Proxy134.createWriter(Unknown Source)
at
org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:138)
... 15 common frames omitted
I believe the incoming flow file should be routed to failure and the output
flow file removed from the session (and its OutputStream closed) in order to be
able to commit the session successfully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
