[
https://issues.apache.org/jira/browse/NIFI-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037190#comment-17037190
]
Elli Schwarz commented on NIFI-7114:
------------------------------------
We are not seeing the same issue with "pipe" as reported above. While I can't
post our exact lsof output, here's basically what it looks like for us:
{{... IPv4 ... ... TCP <url1>:<port> -> <url2>:pcsyn-http (CLOSE_WAIT)}}
This line above, with the same url1 and url2, repeats 17k times! And it keeps
growing larger the longer we keep the Nifi running. There are many many ports.
It seems that these connections are not being closed. I can't find anything
obvious on these site-to-site connections to determine what's going on, other
than to say we have a very large volume of data going through this site-to-site
- but not anywhere near 17,000 flowfiles in one moment.
We currently need to have a cron job that checks if our open files are
approaching our threshold, and the cron job will restart the nifi in that case.
Otherwise we can come back after a weekend (or even overnight!) to several dead
nifis.
> NiFi not closing file handles
> -----------------------------
>
> Key: NIFI-7114
> URL: https://issues.apache.org/jira/browse/NIFI-7114
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 1.10.0, 1.11.0
> Environment: Amazon EC2 running either Amazon Linux 2 or Ubuntu 18.04.
> NiFi has been installed with no change to any configuration file.
> Reporter: Vinicius Zolin
> Priority: Major
> Attachments: destination.xml, lsof.log, lsof.zip, lsofAfter.log,
> lsofBefore.log, openFiles.xlsx, source.xml
>
>
> Since at least version 1.10 NiFi stopped closing file handles. It opens circa
> 500 files per hour (measured using lsof) without any apparent limit until it
> crashes due to too many open files.
>
> Increasing the computer open file limit is not a solution since NiFi will
> still crash, it'll only take longer to do so.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
