Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-08-17 Thread Benjamin Lerer
I. > > Definitely > > > > one of the more impactful performance improvements to the codebase, > > given > > > > the benefits to compaction and memory behaviour. > > > > > > > > From: bened...@apache.org > > > > Date: Wednesday, 2

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-23 Thread Branimir Lambov
e sstables are not copied > > whole, and that a flush is not done at the end. > > > > Regards, > > Branimir > > > > On Wed, Jul 21, 2021 at 4:33 PM bened...@apache.org > > > wrote: > > > > > I would love to help out with this in any way that I ca

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-22 Thread Michael Burman
..@apache.org > wrote: > > > I would love to help out with this in any way that I can, FYI. Definitely > > one of the more impactful performance improvements to the codebase, given > > the benefits to compaction and memory behaviour. > > > > From: bened...@apache

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-21 Thread Branimir Lambov
the benefits to compaction and memory behaviour. > > From: bened...@apache.org > Date: Wednesday, 21 July 2021 at 14:32 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSS] CEP-11: Pluggable memtable implementations > > memtable-as-a-commitlog-index > > Heh, ba

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-21 Thread bened...@apache.org
Subject: Re: [DISCUSS] CEP-11: Pluggable memtable implementations > memtable-as-a-commitlog-index Heh, based on 7282? Yeah, I’ve had this idea for a while now (actually there was a paper that did this a long time ago), and it could be very nice (if for no other benefit than reducing h

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-21 Thread bened...@apache.org
the same concept, however, only that the Memtable must be able to receive an address into a commit log entry and to adopt partial ownership over the entry’s lifecycle. From: Branimir Lambov Date: Wednesday, 21 July 2021 at 14:28 To: dev@cassandra.apache.org Subject: Re: [DISCUSS] CEP-11: Plugga

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-21 Thread Branimir Lambov
> In general, I think we need to make up our mind as to whether we consider the Memtable and CommitLog one logical entity [...], or whether we want to further untangle those two components from an architectural perspective which we started down that road on with the pluggable storage engine

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-21 Thread Michael Burman
Hi, It is nice to see these going forward (and a great use of CEP) so thanks for the proposal. I have my reservations regarding the linking of memtable to CommitLog and flushing and should not leak abstraction from one to another. And I don't see the reasoning why they should be, it doesn't seem t

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-20 Thread Berenguer Blasi
+1. De-tangling, going more modular and clean interfaces sgtm. On 20/7/21 21:45, Nate McCall wrote: > Yay for pluggable memtables!! I havent gone over this in detail yet, but > personally I've always thought integrating something like Arrow would be > cool for sharing data (that's as far as i've g

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-20 Thread Nate McCall
Yay for pluggable memtables!! I havent gone over this in detail yet, but personally I've always thought integrating something like Arrow would be cool for sharing data (that's as far as i've gotten, but anything that makes that kind of experimentation easier would also help with mocking test plumbi

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-20 Thread Jeremiah D Jordan
gt; From: Joshua McKenzie > Date: Tuesday, 20 July 2021 at 18:36 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSS] CEP-11: Pluggable memtable implementations > +1 to the idea. > > In general, I think we need to make up our mind as to whether we consider > the

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-20 Thread bened...@apache.org
evolution of this stuff. From: Joshua McKenzie Date: Tuesday, 20 July 2021 at 18:36 To: dev@cassandra.apache.org Subject: Re: [DISCUSS] CEP-11: Pluggable memtable implementations +1 to the idea. In general, I think we need to make up our mind as to whether we consider the Memtable and CommitLog one

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-20 Thread Joshua McKenzie
d to seeing how the different approaches compare. > > > From: Branimir Lambov > Date: Tuesday, 20 July 2021 at 11:11 > To: dev@cassandra.apache.org > Subject: [DISCUSS] CEP-11: Pluggable memtable implementations > Proposal for a mechanism for plugging in memtable implementations

Re: [DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-20 Thread bened...@apache.org
approaches compare. From: Branimir Lambov Date: Tuesday, 20 July 2021 at 11:11 To: dev@cassandra.apache.org Subject: [DISCUSS] CEP-11: Pluggable memtable implementations Proposal for a mechanism for plugging in memtable implementations: https://cwiki.apache.org/confluence/display/CASSANDRA/CEP

[DISCUSS] CEP-11: Pluggable memtable implementations

2021-07-20 Thread Branimir Lambov
Proposal for a mechanism for plugging in memtable implementations: https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-11%3A+Pluggable+memtable+implementations The proposal supports using custom memtable implementations to support development and testing of improved alternatives, but also en