[
https://issues.apache.org/jira/browse/KUDU-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311993#comment-15311993
]
Todd Lipcon commented on KUDU-1399:
-----------------------------------
8d377ac0fe2be0f9176526e9db745ea6dd70ce86 (KUDU-678) might have helped this a
bit, but it's still probably worth implementing some kind of compaction which
relocates blocks out of containers that only have <5% full or somesuch
> LBM fd usage sometimes doesn't scale right
> ------------------------------------------
>
> Key: KUDU-1399
> URL: https://issues.apache.org/jira/browse/KUDU-1399
> Project: Kudu
> Issue Type: Bug
> Components: fs
> Reporter: Jean-Daniel Cryans
> Assignee: Todd Lipcon
>
> Noticed on a cluster that runs YCSB non-stop in a loop, using a new table
> each time and deleting an old one, while ensuring that it keeps at least 10
> tables to simulate a "loaded" cluster.
> After a few weeks of running, the cluster's file descriptor usage started
> triggering alerts in the monitoring system. The total data consumption never
> really changes since we keep at most 10 tables. Todd found that we have
> thousands of really small log block manager containers.
> The workaround at the moment is to simply increase the ulimit for open file
> descriptors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
