[
https://issues.apache.org/jira/browse/NIFI-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Gresock updated NIFI-3066:
---------------------------------
Description:
With the following properties in MergeContent, I get this error when I try to
process a flow file (which is 135MB):
Merge Strategy = Bin-Packing Algorithm
Merge Format = ZIP
Attribute Strategy = Keep Only Common Attributes
Correlation Attribute Name = bundle.identifier
Minimum Number of Entries = 200
Maximum Number of Entries = 200
Minimum Group Size = 20 MB
Maximum Group Size = *20 MB*
Max Bin Age = 1 min
Maximum number of Bins = 100
Delimiter Strategy = Filename
Keep Path = false
016-11-18 18:00:13,752 ERROR [Timer-Driven Process Thread-83]
o.a.n.processors.standard.MergeContent
java.lang.IllegalArgumentException: Cannot migrate FlowFiles from a Process
Session to itself
at
org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1091)
~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processor.util.bin.Bin.offer(Bin.java:142)
~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.processor.util.bin.BinFiles.binFlowFiles(BinFiles.java:282)
~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:178)
~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089)
~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_111]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_111]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_111]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_111]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_111]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
2016-11-18 18:00:13,753 WARN [Timer-Driven Process Thread-83]
o.a.n.processors.standard.MergeContent
MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80] Processor
Administratively Yielded for 1 sec due to processing failure
However, other smaller flow files are able to successfully go through the
processor. Not sure if it's related to the size of the flow file or not.
Perhaps there just needs to be a check around this line in Bin.java:142, to
make sure we don't try to migrate a flow file to its own session:
session.migrate(getSession(), Collections.singleton(flowFile));
was:
With the following properties in MergeContent, I get this error when I try to
process a flow file (which is 135MB):
<property>
<name>Merge Strategy</name>
<value>Bin-Packing Algorithm</value>
</property>
<property>
<name>Merge Format</name>
<value>ZIP</value>
</property>
<property>
<name>Attribute Strategy</name>
<value>Keep Only Common Attributes</value>
</property>
<property>
<name>Correlation Attribute Name</name>
<value>bundle.identifier</value>
</property>
<property>
<name>Minimum Number of Entries</name>
<value>200</value>
</property>
<property>
<name>Maximum Number of Entries</name>
<value>200</value>
</property>
<property>
<name>Minimum Group Size</name>
<value>20 MB</value>
</property>
<property>
<name>Maximum Group Size</name>
<value>20 MB</value>
</property>
<property>
<name>Max Bin Age</name>
<value>1 min</value>
</property>
<property>
<name>Maximum number of Bins</name>
<value>100</value>
</property>
<property>
<name>Delimiter Strategy</name>
<value>Filename</value>
</property>
<property>
<name>Header File</name>
</property>
<property>
<name>Footer File</name>
</property>
<property>
<name>Demarcator File</name>
</property>
<property>
<name>Compression Level</name>
<value>1</value>
</property>
<property>
<name>Keep Path</name>
<value>false</value>
</property>
016-11-18 18:00:13,752 ERROR [Timer-Driven Process Thread-83]
o.a.n.processors.standard.MergeContent
java.lang.IllegalArgumentException: Cannot migrate FlowFiles from a Process
Session to itself
at
org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1091)
~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at org.apache.nifi.processor.util.bin.Bin.offer(Bin.java:142)
~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.processor.util.bin.BinFiles.binFlowFiles(BinFiles.java:282)
~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:178)
~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089)
~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_111]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_111]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_111]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_111]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_111]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
2016-11-18 18:00:13,753 WARN [Timer-Driven Process Thread-83]
o.a.n.processors.standard.MergeContent
MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80] Processor
Administratively Yielded for 1 sec due to processing failure
However, other smaller flow files are able to successfully go through the
processor. Not sure if it's related to the size of the flow file or not.
> MergeContent: "Cannot migrate FlowFiles from a Process Session to itself"
> -------------------------------------------------------------------------
>
> Key: NIFI-3066
> URL: https://issues.apache.org/jira/browse/NIFI-3066
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.1.0
> Reporter: Joseph Gresock
> Fix For: 1.1.0
>
>
> With the following properties in MergeContent, I get this error when I try to
> process a flow file (which is 135MB):
> Merge Strategy = Bin-Packing Algorithm
> Merge Format = ZIP
> Attribute Strategy = Keep Only Common Attributes
> Correlation Attribute Name = bundle.identifier
> Minimum Number of Entries = 200
> Maximum Number of Entries = 200
> Minimum Group Size = 20 MB
> Maximum Group Size = *20 MB*
> Max Bin Age = 1 min
> Maximum number of Bins = 100
> Delimiter Strategy = Filename
> Keep Path = false
> 016-11-18 18:00:13,752 ERROR [Timer-Driven Process Thread-83]
> o.a.n.processors.standard.MergeContent
> java.lang.IllegalArgumentException: Cannot migrate FlowFiles from a Process
> Session to itself
> at
> org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1091)
> ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at org.apache.nifi.processor.util.bin.Bin.offer(Bin.java:142)
> ~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at
> org.apache.nifi.processor.util.bin.BinFiles.binFlowFiles(BinFiles.java:282)
> ~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at
> org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:178)
> ~[nifi-processor-utils-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089)
> ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_111]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_111]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
> 2016-11-18 18:00:13,753 WARN [Timer-Driven Process Thread-83]
> o.a.n.processors.standard.MergeContent
> MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80] Processor
> Administratively Yielded for 1 sec due to processing failure
> However, other smaller flow files are able to successfully go through the
> processor. Not sure if it's related to the size of the flow file or not.
> Perhaps there just needs to be a check around this line in Bin.java:142, to
> make sure we don't try to migrate a flow file to its own session:
> session.migrate(getSession(), Collections.singleton(flowFile));
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
