[
https://issues.apache.org/jira/browse/NIFI-11924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi resolved NIFI-11924.
------------------------------------
Fix Version/s: 2.0.0
1.24.0
Resolution: Fixed
> Closing HDFS file system in resource provider after using
> ---------------------------------------------------------
>
> Key: NIFI-11924
> URL: https://issues.apache.org/jira/browse/NIFI-11924
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Simon Bence
> Assignee: Simon Bence
> Priority: Critical
> Fix For: 2.0.0, 1.24.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The {{HDFSExternalResourceProvider}} does not close the HDFS {{FileSystem}}
> after usage. This can lead to memory leakage. This needs to be solved.
> Possible solutions:
> I.: Maintain a single {{FileSystem}} resource during the lifecycle of the
> provider and monitor its health
> II.: Open and close a new {{FileSystem}} evert time the provider connects
> with the HDFS
> The first approach is considered a more error prone one as the Provider is
> active during the whole runtime of NiFi, thus additional logic would be
> needed to monitor the health of the resource and replace it in case of need.
> The second approach results a simpler code on the price of the performance:
> opening a new connection every time is subpar compared to the first option
> and in case of a processor which potentially works with high number of flow
> files within a short amount of time would be an important factor. The usage
> of the provider hovewer comes with a small number of calls, not resulting
> significant impact on the performance on NiFi nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
