Re: Effect of frequent mutations / memtable

2017-05-26 Thread Jonathan Haddad
If you're doing high volumes of writes that simply overwrite values, you're going to see memtables flush to disk when the commit log hits it's space limit and you recycle commit log segments. I agree, it makes sense to not write these values to disk only to compact them, if this is your pattern.

Re: Effect of frequent mutations / memtable

2017-05-26 Thread Jan Algermissen
Jonathan, On 26 May 2017, at 17:00, Jonathan Haddad wrote: If you have a small amount of hot data, enable the row cache. The memtable is not designed to be a cache. You will not see a massive performance impact of writing one to disk. Sstables will be in your page cache, meaning you won't

Re: Effect of frequent mutations / memtable

2017-05-26 Thread Jonathan Haddad
If you have a small amount of hot data, enable the row cache. The memtable is not designed to be a cache. You will not see a massive performance impact of writing one to disk. Sstables will be in your page cache, meaning you won't be hitting disk very often. On Fri, May 26, 2017 at 7:41 AM Max C

Re: Effect of frequent mutations / memtable

2017-05-26 Thread Max C
In my case, we're using Cassandra to store QA test data — so the pattern is that we may do a bunch of updates within a few minutes / hours, and then the data will essentially be read-only for the rest of its lifetime (years). My question is the same — do we need to worry about the performance

Re: Effect of frequent mutations / memtable

2017-05-25 Thread Thakrar, Jayesh
That's because Zookeeper is purpose built for such a kind of usage. Its asynchronous nature - e.g. you can create "watchers" with callbacks so that when ephemeral nodes die/disappear (due to servers crashing) makes it better to program. It also reduces the "checkin" and "polling" cycle

Re: Effect of frequent mutations / memtable

2017-05-25 Thread Jan Algermissen
Hi Jayesh, On 25 May 2017, at 18:31, Thakrar, Jayesh wrote: Hi Jan, I would suggest looking at using Zookeeper for such a usecase. thanks - yes, it is an alternative. Out of curiosity: since both, Zk and C* implement Paxos to enable such kind of thing, why do you think Zookeeper would be

Re: Effect of frequent mutations / memtable

2017-05-25 Thread Thakrar, Jayesh
Hi Jan, I would suggest looking at using Zookeeper for such a usecase. See http://zookeeper.apache.org/doc/trunk/recipes.html for some examples. Zookeeper is used for such purposes in Apache HBase (active master), Apache Kafka (active controller), Apache Hadoop, etc. Look for the "Leader