[jira] [Updated] (KUDU-1830) Reduce Kudu WAL log disk usage
[ https://issues.apache.org/jira/browse/KUDU-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adar Dembo updated KUDU-1830: - Issue Type: Sub-task (was: Improvement) Parent: KUDU-1967 > Reduce Kudu WAL log disk usage > -- > > Key: KUDU-1830 > URL: https://issues.apache.org/jira/browse/KUDU-1830 > Project: Kudu > Issue Type: Sub-task > Components: consensus, log >Reporter: Juan Yu > Labels: data-scalability > > WAL log can take significent disk space. So far there are some config to > limit it. but it can go very high. > WAL log size = #tablets * log_segment_size_mb * log segments (1 if there is > write ops to this tablet, can go up to log_max_segments_to_retain) > Logs are retained even if there is no write for a while. > We could reduce the WAL log usage by > - reduce min_segments_to_retain to 1 instead of 2, a > - reduce steady state consumption of idle tablets, roll a WAL if it has had > no writes for a few minutes and size more than a MB or two so that "idle" > tablets have 0 WAL space consumed -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KUDU-1830) Reduce Kudu WAL log disk usage
[ https://issues.apache.org/jira/browse/KUDU-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated KUDU-1830: -- Target Version/s: 1.4.0 > Reduce Kudu WAL log disk usage > -- > > Key: KUDU-1830 > URL: https://issues.apache.org/jira/browse/KUDU-1830 > Project: Kudu > Issue Type: Improvement > Components: consensus, log >Reporter: Juan Yu > Labels: data-scalability > > WAL log can take significent disk space. So far there are some config to > limit it. but it can go very high. > WAL log size = #tablets * log_segment_size_mb * log segments (1 if there is > write ops to this tablet, can go up to log_max_segments_to_retain) > Logs are retained even if there is no write for a while. > We could reduce the WAL log usage by > - reduce min_segments_to_retain to 1 instead of 2, a > - reduce steady state consumption of idle tablets, roll a WAL if it has had > no writes for a few minutes and size more than a MB or two so that "idle" > tablets have 0 WAL space consumed -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KUDU-1830) Reduce Kudu WAL log disk usage
[ https://issues.apache.org/jira/browse/KUDU-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated KUDU-1830: -- Labels: data-scalability (was: ) > Reduce Kudu WAL log disk usage > -- > > Key: KUDU-1830 > URL: https://issues.apache.org/jira/browse/KUDU-1830 > Project: Kudu > Issue Type: Improvement > Components: consensus, log >Reporter: Juan Yu > Labels: data-scalability > > WAL log can take significent disk space. So far there are some config to > limit it. but it can go very high. > WAL log size = #tablets * log_segment_size_mb * log segments (1 if there is > write ops to this tablet, can go up to log_max_segments_to_retain) > Logs are retained even if there is no write for a while. > We could reduce the WAL log usage by > - reduce min_segments_to_retain to 1 instead of 2, a > - reduce steady state consumption of idle tablets, roll a WAL if it has had > no writes for a few minutes and size more than a MB or two so that "idle" > tablets have 0 WAL space consumed -- This message was sent by Atlassian JIRA (v6.3.15#6346)
