Todd Lipcon created KUDU-2856:
---------------------------------
Summary: Track memory and potentially limit count of open
CFileReaders
Key: KUDU-2856
URL: https://issues.apache.org/jira/browse/KUDU-2856
Project: Kudu
Issue Type: Improvement
Components: tserver
Reporter: Todd Lipcon
Currently, CFileReaders are lazy-initted on first access. But, even a
non-initted CFileReader takes some memory (104 bytes currently), and initted
ones never get de-initted. So, for long-running tservers with lots of data, in
the limit, all cfiles will eventually be initted.
In one production tserver I'm looking at, there are 24 million CFileReaders
(2385MB of heap) of which about 3.9 million are initted (about 1.1GB of heap).
If we project this out and assume that eventually all CFileReaders get initted,
we'd be up to about 7GB of heap for initted CFileReaders.
We should consider whether we can make the allocation of CFileReaders more
lazy, and make it possible to deinit and deallocate readers that have not been
recently/frequently used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
