Re: Proposal - GroupCommitLogService

That's exactly the motivation of this proposal. Batch can group only writes which are not persisted (not kept waiting) at that time. In Batch, a write of commitlog isn't kept waiting because the thread lock (the semaphore in 2.2 and 3.0) for sync is released immediately. So, the window size

Re: Proposal - GroupCommitLogService

Does that mean that Batch is not working as designed? If there are other pending writes, Batch should also group them together. (Did you test with giving Batch the same window size as Group?) On Sat, May 13, 2017 at 10:08 AM, Yuji Ito wrote: > Batch outperforms when

Re: Integrating vendor-specific code and developing plugins

> It may be better to figure out how to foster a plugin ecosystem, which is a > bit better than "there's an API go deal with it". This is what Spark is > doing and it seems like a pretty reasonable approach to me: > https://spark-packages.org/ > In thinking about this a bit, we have: Mesos, Beam

Re: Proposal - GroupCommitLogService

Batch outperforms when there is no concurrency. Because GroupCommit should wait the window time, the throughput and the latency are worse in a single request. GroupCommit can gather the commitlog writes which are requested in the window time. Actually, the throughput of a single thread was bounded

Re: Proposal - GroupCommitLogService

Can we replace Batch entirely with this, or are there situations where Batch would outperform (in latency, for instance)? On Sat, May 13, 2017 at 7:21 AM, Yuji Ito wrote: > Hi dev, > > I propose a new CommitLogService, GroupCommitLogService, to improve the > throughput

Re: Integrating vendor-specific code and developing plugins

In accordance with the idea that the codebase should be better tested, it seems to me like things shouldn't be added that aren't testable. If there's a million unit tests that are insanely comprehensive but for some reason can never be run, they serve exactly the same value as no tests. It may

Re: Proposal - GroupCommitLogService

Thanks Jeremiah, I've opened a ticket on JIRA. https://issues.apache.org/jira/browse/CASSANDRA-13530 Best, Yuji On Sat, May 13, 2017 at 9:38 PM, J. D. Jordan wrote: > Sounds interesting. You should open a JIRA and attach your code for > discussion of it. > >

Re: Proposal - GroupCommitLogService

Sounds interesting. You should open a JIRA and attach your code for discussion of it. https://issues.apache.org/jira/browse/CASSANDRA/ -Jeremiah > On May 13, 2017, at 7:21 AM, Yuji Ito wrote: > > Hi dev, > > I propose a new CommitLogService, GroupCommitLogService, to

Proposal - GroupCommitLogService

Hi dev, I propose a new CommitLogService, GroupCommitLogService, to improve the throughput when lots of requests are received. It improved the throughput by maximum 94%. I'd like to discuss about this CommitLogService. Currently, we can select either 2 CommitLog services; Periodic and Batch. In