[
https://issues.apache.org/jira/browse/KUDU-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhangZhen updated KUDU-2226:
----------------------------
Summary: Tablets with too many DRSs will cause a huge DMS memory overhead
(was: Frequently updated table does not flush DeltaMemStore in time and will
occupy a lot of memory)
> Tablets with too many DRSs will cause a huge DMS memory overhead
> ----------------------------------------------------------------
>
> Key: KUDU-2226
> URL: https://issues.apache.org/jira/browse/KUDU-2226
> Project: Kudu
> Issue Type: Improvement
> Affects Versions: 1.3.0
> Environment: CentOS6.5 Linux 2.6.32-431
> Kudu1.3.0
> GitCommit 00813f96b9cb
> Reporter: ZhangZhen
>
> I have a table with 10M rows in total and has been hash partitioned to 16
> buckets. Each tablet has about 100MB on disk size according to the /tablets
> Web UI. Everyday 50K new rows will be inserted into this table, and about 5M
> rows of this table will be updated, that's about half of rows in total, each
> row will be updated only once.
> Then I found something strange, from the /mem-trackers UI of TS, I found
> every tablet of this table occupied about 900MB memory, mainly occupied by
> DeltaMemStore, the peak memory consumption is about 1.8G.
> I don't understand why the DeltaMemStore will cost so much memory, 900MB DMS
> vs 100MB on disk size, that seems strange to me. What's more, I found these
> DMS will be flushed very slowly, so for a long time these memory are
> occupied, which cause "Soft memory limit exceeded" in the TS, and in result
> cause "Rejecting consensus request".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
