[
https://issues.apache.org/jira/browse/NIFI-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037341#comment-17037341
]
Matt Burgess commented on NIFI-7114:
------------------------------------
There might be bug(s) in Jython such as https://bugs.jython.org/issue2420 (not
to say that this is the issue), it might be better to take Jython out of the
equation and either port to Groovy or try existing processors. For example,
instead of SplitJson -> EvaluateJSONPath -> RouteOnAttribute, you could use
QueryRecord (and perhaps a split later on only if necessary) or PartitionRecord
-> RouteOnAttribute (again with a split afterward only if necessary). The
scripts seem take certain attributes and fill the output with them, you might
be able to do that with some combination of AttributesToJSON and/or
JoltTransformJSON / JoltTransformRecord. The final script that merges the JSON
back together looks like it could be replaced with MergeRecord. Not sure any of
this would help with the open files, but would definitely help with overall
performance.
I haven't been able to reproduce with the sample templates yet, but I haven't
run long enough yet, will post any findings here.
> NiFi not closing file handles
> -----------------------------
>
> Key: NIFI-7114
> URL: https://issues.apache.org/jira/browse/NIFI-7114
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 1.10.0, 1.11.0
> Environment: Amazon EC2 running either Amazon Linux 2 or Ubuntu 18.04.
> NiFi has been installed with no change to any configuration file.
> Reporter: Vinicius Zolin
> Priority: Major
> Attachments: destination.xml, lsof.log, lsof.zip, lsofAfter.log,
> lsofBefore.log, openFiles.xlsx, source.xml
>
>
> Since at least version 1.10 NiFi stopped closing file handles. It opens circa
> 500 files per hour (measured using lsof) without any apparent limit until it
> crashes due to too many open files.
>
> Increasing the computer open file limit is not a solution since NiFi will
> still crash, it'll only take longer to do so.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
