[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-8760:
---
Status: Open (was: Patch Available)
> VolatileContentRepository fails to retrieve content from claims with several
> processors
> ---
>
> Key: NIFI-8760
> URL: https://issues.apache.org/jira/browse/NIFI-8760
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.15.0, 1.13.2, 1.13.1, 1.14.0
>Reporter: Matthieu RÉ
>Priority: Major
> Labels: content-repository, volatile
> Attachments:
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch,
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch, flow.xml.gz,
> nifi.properties
>
>
> For several processors such as MergeRecord, QueryRecord, SplitJson, the use
> of VolatileContentRepository implementation infers errors while retrieving
> Flowfiles from claims. The following logs are generated using NiFi 1.13.1
> from Docker and the flow.xml.gz and nifi.properties file attached.
> MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
> configuration):
> {{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
> o.a.nifi.processors.standard.MergeRecord
> MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
> StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655]
> due to org.apache.nifi.controller.repository.ContentNotFoundException: Could
> not find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
> {{at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
> {{at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
> {{at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
> {{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
> {{at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
> {{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
> {{at java.lang.Thread.run(Thread.java:748)}}
> QueryRecord:
> {{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine
> Record Schema from
> StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=2, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
> routing to failure:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
>
[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8760:
---
Fix Version/s: (was: 1.15.0)
> VolatileContentRepository fails to retrieve content from claims with several
> processors
> ---
>
> Key: NIFI-8760
> URL: https://issues.apache.org/jira/browse/NIFI-8760
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0, 1.13.1, 1.13.2, 1.15.0
>Reporter: Matthieu RÉ
>Priority: Major
> Labels: content-repository, volatile
> Attachments:
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch,
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch, flow.xml.gz,
> nifi.properties
>
>
> For several processors such as MergeRecord, QueryRecord, SplitJson, the use
> of VolatileContentRepository implementation infers errors while retrieving
> Flowfiles from claims. The following logs are generated using NiFi 1.13.1
> from Docker and the flow.xml.gz and nifi.properties file attached.
> MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
> configuration):
> {{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
> o.a.nifi.processors.standard.MergeRecord
> MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
> StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655]
> due to org.apache.nifi.controller.repository.ContentNotFoundException: Could
> not find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
> {{at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
> {{at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
> {{at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
> {{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
> {{at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
> {{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
> {{at java.lang.Thread.run(Thread.java:748)}}
> QueryRecord:
> {{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine
> Record Schema from
> StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=2, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
> routing to failure:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
>
[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthieu RÉ updated NIFI-8760:
--
Attachment:
0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch
Fix Version/s: 1.15.0
Status: Patch Available (was: Open)
Hello,
As said in the last commentary, I propose one solution (the easiest) to fix the
bug but maybe a bigger discussion can be starter in order to improve the
management of the VolatileContentRepository.
For now this fixes the issue, I applied it successfully on each branch failing
before.
> VolatileContentRepository fails to retrieve content from claims with several
> processors
> ---
>
> Key: NIFI-8760
> URL: https://issues.apache.org/jira/browse/NIFI-8760
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2, 1.13.1, 1.14.0, 1.15.0
>Reporter: Matthieu RÉ
>Priority: Major
> Labels: content-repository, volatile
> Fix For: 1.15.0
>
> Attachments:
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch,
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch, flow.xml.gz,
> nifi.properties
>
>
> For several processors such as MergeRecord, QueryRecord, SplitJson, the use
> of VolatileContentRepository implementation infers errors while retrieving
> Flowfiles from claims. The following logs are generated using NiFi 1.13.1
> from Docker and the flow.xml.gz and nifi.properties file attached.
> MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
> configuration):
> {{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
> o.a.nifi.processors.standard.MergeRecord
> MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
> StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655]
> due to org.apache.nifi.controller.repository.ContentNotFoundException: Could
> not find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
> {{at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
> {{at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
> {{at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
> {{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
> {{at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
> {{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
> {{at java.lang.Thread.run(Thread.java:748)}}
> QueryRecord:
> {{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine
[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthieu RÉ updated NIFI-8760:
--
Flags: Patch
Affects Version/s: 1.15.0
1.14.0
> VolatileContentRepository fails to retrieve content from claims with several
> processors
> ---
>
> Key: NIFI-8760
> URL: https://issues.apache.org/jira/browse/NIFI-8760
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0, 1.13.1, 1.13.2, 1.15.0
>Reporter: Matthieu RÉ
>Priority: Major
> Labels: content-repository, volatile
> Attachments:
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch, flow.xml.gz,
> nifi.properties
>
>
> For several processors such as MergeRecord, QueryRecord, SplitJson, the use
> of VolatileContentRepository implementation infers errors while retrieving
> Flowfiles from claims. The following logs are generated using NiFi 1.13.1
> from Docker and the flow.xml.gz and nifi.properties file attached.
> MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
> configuration):
> {{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
> o.a.nifi.processors.standard.MergeRecord
> MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
> StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655]
> due to org.apache.nifi.controller.repository.ContentNotFoundException: Could
> not find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
> {{at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
> {{at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
> {{at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
> {{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
> {{at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
> {{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
> {{at java.lang.Thread.run(Thread.java:748)}}
> QueryRecord:
> {{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine
> Record Schema from
> StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=2, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
> routing to failure:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
>
[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthieu RÉ updated NIFI-8760:
--
Attachment: 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch
> VolatileContentRepository fails to retrieve content from claims with several
> processors
> ---
>
> Key: NIFI-8760
> URL: https://issues.apache.org/jira/browse/NIFI-8760
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.1, 1.13.2
>Reporter: Matthieu RÉ
>Priority: Major
> Labels: content-repository, volatile
> Attachments:
> 0001-fix-2-set-VolatileContentRepository-as-non-supportiv.patch, flow.xml.gz,
> nifi.properties
>
>
> For several processors such as MergeRecord, QueryRecord, SplitJson, the use
> of VolatileContentRepository implementation infers errors while retrieving
> Flowfiles from claims. The following logs are generated using NiFi 1.13.1
> from Docker and the flow.xml.gz and nifi.properties file attached.
> MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
> configuration):
> {{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
> o.a.nifi.processors.standard.MergeRecord
> MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
> StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655]
> due to org.apache.nifi.controller.repository.ContentNotFoundException: Could
> not find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
> {{at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
> {{at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
> {{at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
> {{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
> {{at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
> {{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
> {{at java.lang.Thread.run(Thread.java:748)}}
> QueryRecord:
> {{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine
> Record Schema from
> StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=2, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
> routing to failure:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=2,
[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthieu RÉ updated NIFI-8760:
--
Description:
For several processors such as MergeRecord, QueryRecord, SplitJson, the use of
VolatileContentRepository implementation infers errors while retrieving
Flowfiles from claims. The following logs are generated using NiFi 1.13.1 from
Docker and the flow.xml.gz and nifi.properties file attached.
MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
configuration):
{{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
o.a.nifi.processors.standard.MergeRecord
MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0,
length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655] due
to org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0, length=-1]:
org.apache.nifi.controller.repository.ContentNotFoundException: Could not find
content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=6,
container=in-memory, section=section], offset=0, length=-1]}}
{{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0, length=-1]}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
{{at
org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
{{at
org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
{{at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
{{at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
{{at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
{{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
QueryRecord:
{{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
o.a.nifi.processors.standard.QueryRecord
QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine Record
Schema from
StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=2, container=in-memory,
section=section], offset=0,
length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
routing to failure:
org.apache.nifi.controller.repository.ContentNotFoundException: Could not find
content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=2,
container=in-memory, section=section], offset=0, length=-1]}}
{{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=2, container=in-memory,
section=section], offset=0, length=-1]}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
{{at
[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthieu RÉ updated NIFI-8760:
--
Environment: (was: Linux, Docker, not tested on Windows and others)
> VolatileContentRepository fails to retrieve content from claims with several
> processors
> ---
>
> Key: NIFI-8760
> URL: https://issues.apache.org/jira/browse/NIFI-8760
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.1, 1.13.2
>Reporter: Matthieu RÉ
>Priority: Major
> Labels: content-repository, volatile
> Attachments: flow.xml.gz, nifi.properties
>
>
> For several processors such as MergeRecord, QueryRecord, SplitJson, the use
> of Volatile implementations infers errors retrieving Flowfiles from claims.
> The following logs are generated using NiFi 1.13.1 from Docker and the
> flow.xml.gz and nifi.properties file attached.
> MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
> configuration):
> {{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
> o.a.nifi.processors.standard.MergeRecord
> MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
> StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655]
> due to org.apache.nifi.controller.repository.ContentNotFoundException: Could
> not find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=6, container=in-memory,
> section=section], offset=0, length=-1]}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
> {{at
> org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
> {{at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
> {{at
> org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
> {{at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
> {{at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
> {{at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
> {{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
> {{at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
> {{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
> {{at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
> {{at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
> {{at java.lang.Thread.run(Thread.java:748)}}
> QueryRecord:
> {{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine
> Record Schema from
> StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=2, container=in-memory,
> section=section], offset=0,
> length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
> routing to failure:
> org.apache.nifi.controller.repository.ContentNotFoundException: Could not
> find content for StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=2, container=in-memory,
> section=section], offset=0, length=-1]}}
>
[jira] [Updated] (NIFI-8760) VolatileContentRepository fails to retrieve content from claims with several processors
[
https://issues.apache.org/jira/browse/NIFI-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthieu RÉ updated NIFI-8760:
--
Description:
For several processors such as MergeRecord, QueryRecord, SplitJson, the use of
Volatile implementations infers errors retrieving Flowfiles from claims. The
following logs are generated using NiFi 1.13.1 from Docker and the flow.xml.gz
and nifi.properties file attached.
MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default
configuration):
{{2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1]
o.a.nifi.processors.standard.MergeRecord
MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin
StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0,
length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655] due
to org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0, length=-1]:
org.apache.nifi.controller.repository.ContentNotFoundException: Could not find
content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=6,
container=in-memory, section=section], offset=0, length=-1]}}
{{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=6, container=in-memory,
section=section], offset=0, length=-1]}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
{{at
org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)}}
{{at
org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)}}
{{at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)}}
{{at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)}}
{{at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)}}
{{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
QueryRecord:
{{2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4]
o.a.nifi.processors.standard.QueryRecord
QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine Record
Schema from
StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=2, container=in-memory,
section=section], offset=0,
length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655];
routing to failure:
org.apache.nifi.controller.repository.ContentNotFoundException: Could not find
content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=2,
container=in-memory, section=section], offset=0, length=-1]}}
{{org.apache.nifi.controller.repository.ContentNotFoundException: Could not
find content for StandardContentClaim
[resourceClaim=StandardResourceClaim[id=2, container=in-memory,
section=section], offset=0, length=-1]}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)}}
{{at
org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)}}
{{at
