Simon Bence created NIFI-11924:
----------------------------------
Summary: Closing HDFS file system in resource provider after using
Key: NIFI-11924
URL: https://issues.apache.org/jira/browse/NIFI-11924
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Simon Bence
Assignee: Simon Bence
The {{HDFSExternalResourceProvider}} does not close the HDFS {{FileSystem}}
after usage. This can lead to memory leakage. This needs to be solved. Possible
solutions:
I.: Maintain a single {{FileSystem}} resource during the lifecycle of the
provider and monitor its health
II.: Open and close a new {{FileSystem}} evert time the provider connects with
the HDFS
The first approach is considered a more error prone one as the Provider is
active during the whole runtime of NiFi, thus additional logic would be needed
to monitor the health of the resource and replace it in case of need. The
second approach results a simpler code on the price of the performance: opening
a new connection every time is subpar compared to the first option and in case
of a processor which potentially works with high number of flow files within a
short amount of time would be an important factor. The usage of the provider
hovewer comes with a small number of calls, not resulting significant impact on
the performance on NiFi nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
