[
https://issues.apache.org/jira/browse/KUDU-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273716#comment-16273716
]
Jean-Daniel Cryans commented on KUDU-2226:
------------------------------------------
There's probably a big backlog, will be interesting to see if it gets to DMS
flushes or compactions after some time.
> Frequently updated table does not flush DeltaMemStore in time and will occupy
> a lot of memory
> ---------------------------------------------------------------------------------------------
>
> Key: KUDU-2226
> URL: https://issues.apache.org/jira/browse/KUDU-2226
> Project: Kudu
> Issue Type: Improvement
> Affects Versions: 1.3.0
> Environment: CentOS6.5 Linux 2.6.32-431
> Kudu1.3.0
> GitCommit 00813f96b9cb
> Reporter: ZhangZhen
>
> I have a table with 10M rows in total and has been hash partitioned to 16
> buckets. Each tablet has about 100MB on disk size according to the /tablets
> Web UI. Everyday 50K new rows will be inserted into this table, and about 5M
> rows of this table will be updated, that's about half of rows in total, each
> row will be updated only once.
> Then I found something strange, from the /mem-trackers UI of TS, I found
> every tablet of this table occupied about 900MB memory, mainly occupied by
> DeltaMemStore, the peak memory consumption is about 1.8G.
> I don't understand why the DeltaMemStore will cost so much memory, 900MB DMS
> vs 100MB on disk size, that seems strange to me. What's more, I found these
> DMS will be flushed very slowly, so for a long time these memory are
> occupied, which cause "Soft memory limit exceeded" in the TS, and in result
> cause "Rejecting consensus request".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
