David Robinson created MESOS-6267:
-------------------------------------
Summary: Mesos master should periodically compact the replicated
log
Key: MESOS-6267
URL: https://issues.apache.org/jira/browse/MESOS-6267
Project: Mesos
Issue Type: Improvement
Components: replicated log
Reporter: David Robinson
Priority: Minor
The mesos master does not periodically compact the replicated log's leveldb, it
grows without bound. Compaction only occurs on startup, therefore in large
clusters (10k+ agents) where the master is restarted infrequently compaction at
startup can take 2+ minutes to complete. At Twitter we've seen the DB grow to
170+ GB in less than a month and compaction take > 2 minutes. Compaction
reduces storage usage from 160 GB to roughly 50 MB.
{noformat}
I0928 06:06:20.776908 19725 leveldb.cpp:174] Opened db in 763.082271ms
I0928 06:08:41.636145 19725 leveldb.cpp:181] Compacted db in
2.34765004808333mins
{noformat}
The replicated log should not grow without bounds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
