[
https://issues.apache.org/jira/browse/KUDU-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke updated KUDU-2318:
------------------------------
Component/s: perf
> Kudu flushes aggressively even if it can't relieve memory pressure.
> -------------------------------------------------------------------
>
> Key: KUDU-2318
> URL: https://issues.apache.org/jira/browse/KUDU-2318
> Project: Kudu
> Issue Type: Bug
> Components: perf
> Affects Versions: 1.6.0
> Reporter: William Berkeley
> Priority: Major
> Attachments: rowsetblackbarofdeath.png
>
>
> Kudu starts flushing aggressively when memory usage exceeds 60% of the hard
> limit. In pathological cases, this can cause Kudu to flush an extremely large
> amount of small rowsets.
> For example, if -block_cache_capacity_mb is set to over 60% of the hard
> limit, and the block cache fills, then Kudu will always be under memory
> pressure. Flushing won't ever be able to reduce memory usage under the
> aggressive flushing threshold. However, Kudu will still flush, producing lots
> of tiny rowsets. This can eventually cause problems like KUDU-2317.
> Attached is the rowset diagram from a tablet showing this phenomenon. To
> produce this I
> 1. Ran the tablet servers with -memory_limit_hard_bytes=1048576000 (1GiB) and
> -block_cache_capacity_mb=750.
> 2. Started an insert workload (single-threaded so insert would be
> sequential): build/latest/bin/kudu perf loadgen -keep_auto_table
> -num_threads=1 -num_rows_per_thread=100000000.
> 3. Waited a bit for tablet servers to have a gigabyte or two of data.
> 4. Ran a ksck checksum set to cache blocks: build/latest/bin/kudu cluster
> ksck -checksum_scan -checksum_cache_blocks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
