[
https://issues.apache.org/jira/browse/HBASE-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16045581#comment-16045581
]
Allan Yang commented on HBASE-18203:
------------------------------------
It is a very useful feature. We often encounter "open too many files" exception
if storefile number is too many on a single RS. the RS process holds too many
handlers(tcp connections) and grows unboundedly with storefile numbers.
> Intelligently manage a pool of open references to store files
> -------------------------------------------------------------
>
> Key: HBASE-18203
> URL: https://issues.apache.org/jira/browse/HBASE-18203
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 2.0.0
> Reporter: Andrew Purtell
>
> When bringing a region online we open every store file and keep the file
> open, to avoid further round trips to the HDFS namenode during reads. Naively
> keeping open every store file we encounter is a bad idea. There should be an
> upper bound. We should close and reopen files as needed once we are above the
> upper bound. We should choose candidates to close on a LRU basis. Otherwise
> we can (and some users have in production) overrun high (~64k) open file
> handle limits on the server if the aggregate number of store files is too
> large.
> Note the 'open files' here refers to open/active references to files at the
> HDFS level. How this maps to active file descriptors at the OS level depends
> on concurrency of access (block transfers, short circuit reads). The more
> open files we have at the HDFS level the higher number of OS level file
> handles we can expect to consume.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
