[
https://issues.apache.org/jira/browse/NIFI-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wesley L Lawrence updated NIFI-4064:
------------------------------------
Description:
I have a simple Flow where a file containing JSON data is read, and converted
to Avro.
!FlowWithOneFileInQueue.png!
However, I have the resulting Avro container file parked in a queue going to a
Funnel, and it causes my CPU to spike.
!NifiPreEmptyHighCpu.png!
Emptying the queue causes the CPU to return to normal.
!NifiPostEmptyLowCpu.png!
It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly
determine that the Funnel shouldn't be run, but since there is both a FlowFile
in queue, and a relaiton, it also decides to not yield.
I've added both a fix for this (patch to follow), as well as a new unit test to
make sure Funnels without outgoing connections get yielded. With the fix, the
CPU spiking is gone.
!WithFixPrePostCpu.png!
was:
I have a simple Flow where a file containing JSON data is read, and converted
to Avro.
!FlowWithOneFileInQueue.png|thumbnail!
However, I have the resulting Avro container file parked in a queue going to a
Funnel, and it causes my CPU to spike.
!NifiPreEmptyHighCpu.png!
Emptying the queue causes the CPU to return to normal.
!NifiPostEmptyLowCpu.png!
It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly
determine that the Funnel shouldn't be run, but since there is both a FlowFile
in queue, and a relaiton, it also decides to not yield.
I've added both a fix for this (patch to follow), as well as a new unit test to
make sure Funnels without outgoing connections get yielded. With the fix, the
CPU spiking is gone.
!WithFixPrePostCpu.png!
> Funnel with no outgoing connections and a queued Flow File uses high CPU
> ------------------------------------------------------------------------
>
> Key: NIFI-4064
> URL: https://issues.apache.org/jira/browse/NIFI-4064
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.3.0
> Reporter: Wesley L Lawrence
> Priority: Minor
> Attachments: FlowWithOneFileInQueue.png, NifiPostEmptyLowCpu.png,
> NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png
>
>
> I have a simple Flow where a file containing JSON data is read, and converted
> to Avro.
> !FlowWithOneFileInQueue.png!
> However, I have the resulting Avro container file parked in a queue going to
> a Funnel, and it causes my CPU to spike.
> !NifiPreEmptyHighCpu.png!
> Emptying the queue causes the CPU to return to normal.
> !NifiPostEmptyLowCpu.png!
> It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly
> determine that the Funnel shouldn't be run, but since there is both a
> FlowFile in queue, and a relaiton, it also decides to not yield.
> I've added both a fix for this (patch to follow), as well as a new unit test
> to make sure Funnels without outgoing connections get yielded. With the fix,
> the CPU spiking is gone.
> !WithFixPrePostCpu.png!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
