Mark Payne created NIFI-1920:
--------------------------------
Summary: Poor performance for SplitText when its 'splits'
relationship is cloned
Key: NIFI-1920
URL: https://issues.apache.org/jira/browse/NIFI-1920
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
On my laptop, I downloaded the file
http://standards.ieee.org/develop/regauth/oui/oui.csv and ran it through
SplitText, splitting into 1-line FlowFiles. It completed in about 1.8 seconds.
I then routed the 'splits' relationship to a second processor so that the data
would be cloned. Rather than completing in 1.8 seconds, when I ran the file
through again, it took 2.5 minutes to complete. A stack trace during that time
shows the time being spent adding the 'CLONE' Provenance Event in the
Provenance Reporter:
{code}
"Timer-Driven Process Thread-3" Id=93 RUNNABLE
at java.util.HashMap$TreeNode.find(HashMap.java:1865)
at java.util.HashMap$TreeNode.find(HashMap.java:1861)
at java.util.HashMap$TreeNode.find(HashMap.java:1861)
at java.util.HashMap$TreeNode.find(HashMap.java:1861)
at java.util.HashMap$TreeNode.find(HashMap.java:1861)
at java.util.HashMap$TreeNode.find(HashMap.java:1861)
at java.util.HashMap$TreeNode.find(HashMap.java:1861)
at java.util.HashMap$TreeNode.putTreeVal(HashMap.java:1981)
at java.util.HashMap.putVal(HashMap.java:637)
at java.util.HashMap.put(HashMap.java:611)
at java.util.HashSet.add(HashSet.java:219)
at java.util.AbstractCollection.addAll(AbstractCollection.java:344)
at
org.apache.nifi.controller.repository.StandardProcessSession$Checkpoint.checkpoint(StandardProcessSession.java:2572)
at
org.apache.nifi.controller.repository.StandardProcessSession$Checkpoint.access$100(StandardProcessSession.java:2539)
at
org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:276)
at
org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:282)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1072)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Number of Locked Synchronizers: 1
- java.util.concurrent.ThreadPoolExecutor$Worker@3e0d1518
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
