[
https://issues.apache.org/jira/browse/NIFI-11924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757387#comment-17757387
]
ASF subversion and git services commented on NIFI-11924:
--------------------------------------------------------
Commit 5802db26677464e1297b773e40f65a9dac75a5ac in nifi's branch
refs/heads/support/nifi-1.x from Simon Bence
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5802db2667 ]
NIFI-11924 Closing FileSystem after using in HDFSExternalResourceProvider
This closes #7588.
Signed-off-by: Peter Turcsanyi <[email protected]>
> Closing HDFS file system in resource provider after using
> ---------------------------------------------------------
>
> Key: NIFI-11924
> URL: https://issues.apache.org/jira/browse/NIFI-11924
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Simon Bence
> Assignee: Simon Bence
> Priority: Critical
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The {{HDFSExternalResourceProvider}} does not close the HDFS {{FileSystem}}
> after usage. This can lead to memory leakage. This needs to be solved.
> Possible solutions:
> I.: Maintain a single {{FileSystem}} resource during the lifecycle of the
> provider and monitor its health
> II.: Open and close a new {{FileSystem}} evert time the provider connects
> with the HDFS
> The first approach is considered a more error prone one as the Provider is
> active during the whole runtime of NiFi, thus additional logic would be
> needed to monitor the health of the resource and replace it in case of need.
> The second approach results a simpler code on the price of the performance:
> opening a new connection every time is subpar compared to the first option
> and in case of a processor which potentially works with high number of flow
> files within a short amount of time would be an important factor. The usage
> of the provider hovewer comes with a small number of calls, not resulting
> significant impact on the performance on NiFi nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
