Mark Payne created NIFI-731:
-------------------------------
Summary: If content repo is unable to destroy content as fast as
it is generated, nifi performance becomes very sporatic
Key: NIFI-731
URL: https://issues.apache.org/jira/browse/NIFI-731
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 0.2.0
When the FlowFile Repository marks claims as destructable, it puts the
notification on a queue that the content repo pulls from. If the content repo
cannot keep up, the queue will fill, resulting in backpressure, that prevents
the FlowFile repository from being updated. This, in turn, causes Processors to
block, waiting on space to become available. This is by design.
However, the capacity of this queue is quite large, and the content repo drains
the entire queue, then destroys all content claims that are on it. As a result,
this act of destroying claims can be quite long, and Processors can block for
quite a period of time, leading to very sporadic performance.
Instead, the content repo should pull from the queue and destroy the claims one
at a time or in small batches, instead of draining the entire queue each time.
This should result in much less sporadic behavior.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
