Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-10 Thread Branimir Lambov
cessary? > > > > *From: *Branimir Lambov > *Date: *Wednesday, 9 February 2022 at 09:35 > *To: *dev@cassandra.apache.org > > *Subject: *Re: [DISCUSS] CEP-19: Trie memtable implementation > > If I understand this correctly, you prefer _not_ to have an option to give > t

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-09 Thread Bowen Song
apache.org *Subject: *Re: [DISCUSS] CEP-19: Trie memtable implementation If I understand this correctly, you prefer _not_ to have an option to give the configuration explicitly in the schema. I.e. force the configurations ("templates" in current terms) to be specified in the yaml, a

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-09 Thread bened...@apache.org
Why not have some default templates that can be specified by the schema without touching the yaml, but overridden in the yaml as necessary? From: Branimir Lambov Date: Wednesday, 9 February 2022 at 09:35 To: dev@cassandra.apache.org Subject: Re: [DISCUSS] CEP-19: Trie memtable implementation

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-09 Thread Branimir Lambov
If I understand this correctly, you prefer _not_ to have an option to give the configuration explicitly in the schema. I.e. force the configurations ("templates" in current terms) to be specified in the yaml, and only allow tables to specify which one to use among them? This does sound at least

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-08 Thread Dinesh Joshi
My quick reading of the code suggests that schema will override the operator's default preference in the YAML. In the event of a bug in the new implementation, there could be situation where the operator might need to override this via the YAML. > On Feb 8, 2022, at 12:29 PM, Jeremiah D Jordan

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-08 Thread Jeremiah D Jordan
I don’t really see most users touching the default implementation. I would expect the main reason someone would change would be 1. They run into some bug that is only in one of the implementations. 2. They have persistent memory and so want to use

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-08 Thread bened...@apache.org
for those with an old config file that does not specify an implementation. From: Dinesh Joshi Date: Tuesday, 8 February 2022 at 20:21 To: dev@cassandra.apache.org Subject: Re: [DISCUSS] CEP-19: Trie memtable implementation Thank you for sharing the perf test results. Going back to the schema

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-08 Thread Dinesh Joshi
Thank you for sharing the perf test results. Going back to the schema vs yaml configuration. I am concerned users may pick the wrong implementation for their use-case. Is there any chance for us to automatically pick a MemTable implementation based on heuristics? Do we foresee users ever

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-07 Thread C. Scott Andreas
Branimir, thank you for sharing these results. The numbers are exciting - particularly the UCS test in which compaction keeps up, and your note mentioning 30% larger L0 flushes due to the more compact memory representation. Great work. — Scott > On Feb 7, 2022, at 8:39 AM, Branimir Lambov

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-07 Thread Branimir Lambov
Added some performance results to the ticket: https://issues.apache.org/jira/browse/CASSANDRA-17240 Regards, Branimir On Sat, Feb 5, 2022 at 10:59 PM Dinesh Joshi wrote: > This is excellent. Thanks for opening up this CEP. It would be great to > get some stats around GC allocation rate /

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-02-05 Thread Dinesh Joshi
This is excellent. Thanks for opening up this CEP. It would be great to get some stats around GC allocation rate / memory pressure, read & write latencies, etc. compared to existing implementation. Dinesh > On Jan 18, 2022, at 2:13 AM, Branimir Lambov wrote: > > The memtable pluggability API

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-01-18 Thread Branimir Lambov
The memtable pluggability API (CEP-11) is per-table to enable memtable selection that suits specific workflows. It also makes full sense to permit per-node configuration, both to be able to modify the configuration to suit heterogeneous deployments better, as well as to test changes for

Re: [DISCUSS] CEP-19: Trie memtable implementation

2022-01-14 Thread Jeff Jirsa
Sounds like a great addition Can you share some of the details around gc and latency improvements you’ve observed with the list? Any specific reason the confirmation is through schema vs yaml? Presumably it’s so a user can test per table, but this changes every host in a cluster, so the

[DISCUSS] CEP-19: Trie memtable implementation

2022-01-10 Thread Branimir Lambov
We would like to contribute our TrieMemtable to Cassandra. https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-19%3A+Trie+memtable+implementation This is a new memtable solution aimed to replace the legacy implementation, developed with the following objectives: - lowering the on-heap