[
https://issues.apache.org/jira/browse/KUDU-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon reassigned KUDU-1830:
---------------------------------
Assignee: Todd Lipcon
65c3aec9aeb7cb684cac98add6f48b737ae0de85 reduced the WAL segment size to 8MB
and 7dd30b1c9c2e362efe2b81eb3acccd5c3b003e78 changed to retain only one segment
by default. So, this should reduce the total consumption by ~16x for cold
tablets.
> Reduce Kudu WAL log disk usage
> ------------------------------
>
> Key: KUDU-1830
> URL: https://issues.apache.org/jira/browse/KUDU-1830
> Project: Kudu
> Issue Type: Sub-task
> Components: consensus, log
> Reporter: Juan Yu
> Assignee: Todd Lipcon
> Labels: data-scalability
> Fix For: 1.4.0
>
>
> WAL log can take significent disk space. So far there are some config to
> limit it. but it can go very high.
> WAL log size = #tablets * log_segment_size_mb * log segments (1 if there is
> write ops to this tablet, can go up to log_max_segments_to_retain)
> Logs are retained even if there is no write for a while.
> We could reduce the WAL log usage by
> - reduce min_segments_to_retain to 1 instead of 2, a
> - reduce steady state consumption of idle tablets, roll a WAL if it has had
> no writes for a few minutes and size more than a MB or two so that "idle"
> tablets have 0 WAL space consumed
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
