[
https://issues.apache.org/jira/browse/KUDU-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16291773#comment-16291773
]
Mike Percy commented on KUDU-2238:
----------------------------------
[~jdcryans] you did some investigation into this issue today, what do you think
of using this as a tracking ticket for improvements to DMS flush selection?
Even the rejections and force flushes were not triggering resolution of the
issue in this case. If so we should add more detail about your findings.
> Big DMS not flush under memory pressure
> ---------------------------------------
>
> Key: KUDU-2238
> URL: https://issues.apache.org/jira/browse/KUDU-2238
> Project: Kudu
> Issue Type: Bug
> Affects Versions: 1.3.0
> Environment: CentOS6.5 Linux 2.6.32-431
> Kudu1.3.0
> GitCommit 00813f96b9cb
> Reporter: ZhangZhen
> Attachments: memory_anchored.png, memory_consumed.png
>
>
> I have a table with many updates, its DMS consumes a lot memory and cause
> “Soft Memory Limit Exceed”. I check the /mem-trackers on the tablet server,
> one of its DMS consumes about 3G memory, but check the /maintenance-manager,
> its FlushDeltaMemStoresOp can only free 763B anchored memory and
> perf_improvement is 0. Is this normal? I know Kudu is not optimized for
> updates, but still confused why the DMS won’t be flushed under memory
> pressure.
> Infos from /mem-trackers:
> !memory_consumed.png!
> tablet-5941a8bb934e4686abd1bfff9e35c860 server none 3.00G 3.00G
> txn_tracker tablet-5941a8bb934e4686abd1bfff9e35c860 64.00M 0B
> 1.67M
> MemRowSet-339 tablet-5941a8bb934e4686abd1bfff9e35c860 none 265B
> 265B
> DeltaMemStores tablet-5941a8bb934e4686abd1bfff9e35c860 none 3.00G
> 3.00G
> Infos from /maintenance-manager
> !memory_anchored.png!
> FlushDeltaMemStoresOp(5941a8bb934e4686abd1bfff9e35c860) true 763B
> 511.15M 0
> The tablet 5941a8bb934e4686abd1bfff9e35c860 has 16 RowSets in total
> Some configs of MM:
> --enable_maintenance_manager=true
> --log_target_replay_size_mb=1024
> --maintenance_manager_history_size=8
> --maintenance_manager_num_threads=6
> --maintenance_manager_polling_interval_ms=50
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
