Re: [Proposal] Replicated log storage compaction

2018-07-27 Thread Ilya Pronin
As discussed, I've filed a LevelDB issue: https://github.com/google/leveldb/issues/603. So far it seems that the LevelDB behavior that we see is unexpected. I'll post a patch with the temporary workaround that I described in the first email in https://issues.apache.org/jira/browse/MESOS-184. It

Re: [Proposal] Replicated log storage compaction

2018-07-11 Thread Judith Malnick
Hey Ilya, If you'd like to generate some real-time conversation about your proposal this might be a good thing to talk about during tomorrow's developer sync at 10:00 am Pacific time. If you're interested please feel free to put it on the agenda

Re: [Proposal] Replicated log storage compaction

2018-07-06 Thread Benjamin Mahler
I was chatting with Ilya on slack and I'll re-post here: * Like Jie, I was hoping for a toggle (maybe it should start default off until we have production experience? sounds like Ilya has already experience with it running in test clusters so far) * I was asking whether this would be considered

Re: [Proposal] Replicated log storage compaction

2018-07-06 Thread Vinod Kone
I don't know about the replicated log, but the proposal seems find to me. Jie/BenM, do you guys have an opinion? On Mon, Jul 2, 2018 at 10:57 PM Santhosh Kumar Shanmugham wrote: > +1. Aurora will hugely benefit from this change. > > On Mon, Jul 2, 2018 at 4:49 PM Ilya Pronin > wrote: > > > Hi

Re: [Proposal] Replicated log storage compaction

2018-07-02 Thread Santhosh Kumar Shanmugham
+1. Aurora will hugely benefit from this change. On Mon, Jul 2, 2018 at 4:49 PM Ilya Pronin wrote: > Hi everyone, > > I'd like to propose adding "manual" LevelDB compaction to the > replicated log truncation process. > > Motivation > > Mesos Master and Aurora Scheduler use the replicated log to

[Proposal] Replicated log storage compaction

2018-07-02 Thread Ilya Pronin
Hi everyone, I'd like to propose adding "manual" LevelDB compaction to the replicated log truncation process. Motivation Mesos Master and Aurora Scheduler use the replicated log to persist information about the cluster. This log is periodically truncated to prune outdated log entries. However