[
https://issues.apache.org/jira/browse/NIFI-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-6790.
------------------------------------
Fix Version/s: 1.11.3
Resolution: Fixed
> Load-balanced queues cause huge number of open pipes
> ----------------------------------------------------
>
> Key: NIFI-6790
> URL: https://issues.apache.org/jira/browse/NIFI-6790
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.9.2
> Environment: Ubuntu,OpenJDK8
> Reporter: Matthew Knight
> Priority: Critical
> Labels: Cluster, LoadBalancing
> Fix For: 1.11.3
>
> Attachments: flow_exception.png, lsof_loadbalanced.png,
> lsof_notloadbalanced.png
>
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> Even very basic flows using load balance connecting queues result in huge
> numbers of open pipes which don't seem to get closed and result in the entire
> cluster being brought to a standstill because of "too many open files". I
> tried apples-to-apples with equivalent very basic flows, one load balanced
> and one not, to show how easy this has been to reproduce. I'm attaching some
> screenshots that show the "too many open files" issue in action in my NiFi
> cluster, along with the output of LSOF for the primary node of a cluster with
> vs. without load balancing. When load balancing is enabled there is a huge
> number of 'pipe' and 'eventpoll' items, without load balancing things are a
> bit more balanced.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
