[
https://issues.apache.org/jira/browse/KUDU-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288650#comment-16288650
]
ZhangZhen commented on KUDU-2226:
---------------------------------
After about one week's observation, it seems few compactions and flushes have
been done. I read some source code and find maybe it's my workload that will
not trigger the compaction. Most of the DRSs in my tablet are skinny and not
overlap with each other, so
the CompactRowSetsOp won't get a high perf_improvement and will not be
executed by MM. At last I set flush_threshold_secs to a big number and recreate
this table, so it will not have too many DRSs and the problem is gone.
> Tablets with too many DRSs will cause a huge DMS memory overhead
> ----------------------------------------------------------------
>
> Key: KUDU-2226
> URL: https://issues.apache.org/jira/browse/KUDU-2226
> Project: Kudu
> Issue Type: Improvement
> Affects Versions: 1.3.0
> Environment: CentOS6.5 Linux 2.6.32-431
> Kudu1.3.0
> GitCommit 00813f96b9cb
> Reporter: ZhangZhen
>
> I have a table with 10M rows in total and has been hash partitioned to 16
> buckets. Each tablet has about 100MB on disk size according to the /tablets
> Web UI. Everyday 50K new rows will be inserted into this table, and about 5M
> rows of this table will be updated, that's about half of rows in total, each
> row will be updated only once.
> Then I found something strange, from the /mem-trackers UI of TS, I found
> every tablet of this table occupied about 900MB memory, mainly occupied by
> DeltaMemStore, the peak memory consumption is about 1.8G.
> I don't understand why the DeltaMemStore will cost so much memory, 900MB DMS
> vs 100MB on disk size, that seems strange to me. What's more, I found these
> DMS will be flushed very slowly, so for a long time these memory are
> occupied, which cause "Soft memory limit exceeded" in the TS, and in result
> cause "Rejecting consensus request".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
