Matthieu RÉ created NIFI-8760:
---------------------------------
Summary: VolatileContentRepository fails to retrieve content from
claims with several processors
Key: NIFI-8760
URL: https://issues.apache.org/jira/browse/NIFI-8760
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.13.2, 1.13.1
Environment: Linux, Docker, not tested on Windows and others
Reporter: Matthieu RÉ
Attachments: flow.xml.gz, nifi.properties
For several processors such as MergeRecord, QueryRecord, SplitJson, the use of
Volatile implementations infers errors retrieving Flowfiles from claims. The
following logs are generated using NiFi 1.13.1 from Docker and the flow.xml.gz
and nifi.properties file attached.
MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
configuration):
{{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
o.a.nifi.processors.standard.MergeRecord
MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0,
length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655] due
to org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0, length=-1]:
org.apache.nifi.controller.repository.ContentNotFoundException: Could not find
content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=6,
container=in-memory, section=section], offset=0, length=-1]}}
{{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0, length=-1]}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
{{at
org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
{{at
org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
{{at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
{{at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
{{at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
{{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
QueryRecord:
{{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
o.a.nifi.processors.standard.QueryRecord
QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine Record
Schema from
StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=2, container=in-memory,
section=section], offset=0,
length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
routing to failure:
org.apache.nifi.controller.repository.ContentNotFoundException: Could not find
content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=2,
container=in-memory, section=section], offset=0, length=-1]}}
{{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=2, container=in-memory,
section=section], offset=0, length=-1]}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
{{at
org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:294)}}
{{at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)}}
{{at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
{{at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
{{at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
{{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
SplitJson:
{{2021-07-06 10:15:10,178 ERROR [Timer-Driven Process Thread-5]
o.a.nifi.processors.standard.SplitJson
SplitJson[id=7b411bdc-017a-1000-0f48-53d6a2ad5ee9]
SplitJson[id=7b411bdc-017a-1000-0f48-53d6a2ad5ee9] failed to process session
due to java.lang.NullPointerException; Processor Administratively Yielded for 1
sec: java.lang.NullPointerException}}
{{java.lang.NullPointerException: null}}
{{at
org.apache.nifi.processors.standard.SplitJson.onTrigger(SplitJson.java:199)}}
{{at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)}}
{{at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
{{at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
{{at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
{{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
This issue is not reproducible at 1.13.0 by my side, so it could correlate with
the commit
[528fce2407d092d4ced1a58fcc14d0bc6e660b89|https://github.com/apache/nifi/commit/528fce2407d092d4ced1a58fcc14d0bc6e660b89].
With some support I would be glad to help investigate and solve the issue.
{{{{}}}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
