[
https://issues.apache.org/jira/browse/HDDS-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581961#comment-17581961
]
Wei-Chiu Chuang commented on HDDS-7136:
---------------------------------------
This is called "file handle cache" by Impala which is intended to reduce the
load to metadata layer (for HDFS, that is NameNode)
https://docs.cloudera.com/documentation/enterprise/latest/topics/impala_scalability.html#scalability_file_handle_cache
So yes it caches file handles (which is an input stream) for an extended period
of time. By default, up to 20k file handles can be cached for up to 8 hours.
> Memory leak due to ChunkInputStream.close() not releasing buffer
> ----------------------------------------------------------------
>
> Key: HDDS-7136
> URL: https://issues.apache.org/jira/browse/HDDS-7136
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Priority: Critical
> Labels: pull-request-available
>
> We found Impala queries that run on ofs does not release memory after
> completion.
> Looking at heap dump, we realized unbuffer() was not called (due to
> HDDS-7135) and Impala calls close() on the input stream instead.
> According to InputStream javadoc, the close() should
> bq. "Closes this input stream and releases any system resources associated
> with the stream."
> I believe close() should release buffer in addition to sockets (which it
> already does).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
