Paul Kelly created NIFI-7454:
--------------------------------
Summary: Directory handle leak in ListFile
Key: NIFI-7454
URL: https://issues.apache.org/jira/browse/NIFI-7454
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.11.4
Reporter: Paul Kelly
Attachments: flow.xml.gz
ListFile does not seem to be closing directory handles if an exception is
raised during the directory walk. For example, if ListFile is recursively
polling a folder and the NiFi user does not have permission to list the
contents of one of the subfolders, an "Access Denied" bulletin is raised and
the number of DIR handles opened by the NiFi process as shown by lsof
increases. Eventually, NiFi stops working due to "Too many open files".
This has been tested with NiFi 1.11.4 on Ubuntu 18.04 with Java 11. I'm not
sure in which version this behavior started.
Attached is a sample flow.xml.gz that demonstrates this behavior. A clean copy
of NiFi 1.11.4 was installed with no modifications to any configuration file.
NiFi is running as the user "xubuntu". The permissions on the folders are as
follows:
drwxrwxr-x 3 xubuntu xubuntu 4096 May 15 08:55 /home/xubuntu/test
drwxrwx--- 2 root root 4096 May 15 08:55 /home/xubuntu/test/subtest
NiFi has permission to list the contents of "/home/xubuntu/test", but not
"/home/xubuntu/test/subtest".
The number of open DIR handles is being monitored with the command:
lsof +E -P -p 347813 | tr -s " " | cut -d" " -f5 | sort | uniq -c
where 347813 is the pid of the NiFi process.
DIR handle counts start at 4 immediately after NiFi is launched and steadily
grow to tens of thousands, until file handles are exhausted. Additionally,
these handles do not seem to be released even if the ListFile processor is
stopped.
Please let me know if there is any additional information I can provide.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
