[
https://issues.apache.org/jira/browse/NIFI-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026697#comment-17026697
]
Tamas Palfy commented on NIFI-7076:
-----------------------------------
[~Absolutesantaja], I tested the new revert commit fix and for me it works,
however for a while I updated only 1 of my 3 nodes and seemed the issue is
still there. Are you sure all NiFi instances you are running have been updated?
[~markap14], [~joewitt]
I checked my changes in NIFI-6787 again and I see the issue was that the
checking if the queue is full should have depended on the balancing strategy
(as it is handled so when choosing a partitioner).
So I think a fix for that is fairly simple - just make sure we use the new
"isFull" logic only for round robin balancing strategy.
{quote}
Secondly, even with the above problem addressed, we still would have a
situation where in order for backpressure to be applied, if there are N nodes
in the cluster, we must reach N times the configured backpressure. For example,
if backpressure threshold is at 10,000 FlowFiles and there are 3 nodes, we have
to reach 30,000 FlowFiles in the queue before backpressure is applied on that
node.
{quote}
The adjustment was intended to work with round robin balancing strategy
specifically. In such cases I would think this is more of a feature than a bug.
The more nodes a cluster has the more capacity the overall system has. The
outgoing buffer is basically the sum of the individual buffers of the nodes. As
long as there are nodes that can accept flowfiles, it should be okay for the
processor to run.
> Back Pressure Not Being Applied In Cluster Mode
> -----------------------------------------------
>
> Key: NIFI-7076
> URL: https://issues.apache.org/jira/browse/NIFI-7076
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.11.0
> Reporter: Shawn Weeks
> Assignee: Mark Payne
> Priority: Critical
> Fix For: 1.12.0, 1.11.1
>
> Attachments: Bug_Example.xml, Screen Shot 2020-01-29 at 12.49.17
> PM.png, Screen Shot 2020-01-29 at 12.49.55 PM.png, Screen Shot 2020-01-29 at
> 5.36.42 PM.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In the attached example back pressure does not get applied if executed on a
> NiFi cluster. Single node works fine. I've reproduced this locally on a
> Redhat 7 based cluster with embedded zookeeper and a Kubernetes based cluster
> running external zookeeper.
> This has been reproduced both with the embedded zookeeper with 1.11 and
> external docker based zookeeper from
> docker.io/bitnami/zookeeper:3.5.6-debian-10-r0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
