Re: storing indexes on ssd

2018-02-13 Thread Oleksandr Shulgin
On Tue, Feb 13, 2018 at 10:46 PM, Dan Kinder wrote: > On a single node that's a bit less than half full, the index files are 87G. > That's not small, true. Out of curiosity: how much data per node do you have in total? How will OS disk cache know to keep the index file

Re: storing indexes on ssd

2018-02-13 Thread Jon Haddad
It seems like cart-before-horse decision to assume you want to keep your index files cached but not your data files. Why not rely on lvmcache’s statistics about file access to determine what to keep and what not to? It’s going to keep your most heavily hit blocks in the cache and your least

Re: storing indexes on ssd

2018-02-13 Thread Dan Kinder
On a single node that's a bit less than half full, the index files are 87G. How will OS disk cache know to keep the index file blocks cached but not cache blocks from the data files? As far as I know it is not smart enough to handle that gracefully. Re: ram expensiveness, see

Re: storing indexes on ssd

2018-02-13 Thread Oleksandr Shulgin
On Tue, Feb 13, 2018 at 1:30 AM, Dan Kinder wrote: > Created https://issues.apache.org/jira/browse/CASSANDRA-14229 > This is confusing. You've already started the conversation here... How big are your index files in the end? Even if Cassandra doesn't cache them in or

Re: storing indexes on ssd

2018-02-12 Thread Dan Kinder
Created https://issues.apache.org/jira/browse/CASSANDRA-14229 On Mon, Feb 12, 2018 at 12:10 AM, Mateusz Korniak < mateusz-li...@ant.gliwice.pl> wrote: > On Saturday 10 of February 2018 23:09:40 Dan Kinder wrote: > > We're optimizing Cassandra right now for fairly random reads on a large > >

Re: storing indexes on ssd

2018-02-12 Thread Mateusz Korniak
On Saturday 10 of February 2018 23:09:40 Dan Kinder wrote: > We're optimizing Cassandra right now for fairly random reads on a large > dataset. In this dataset, the values are much larger than the keys. I was > wondering, is it possible to have Cassandra write the *index* files > (*-Index.db) to

Re: storing indexes on ssd

2018-02-11 Thread sankalp kohli
Cassandra does not support this currently. You can create a JIRA and start the conversation On Sat, Feb 10, 2018 at 11:09 PM, Dan Kinder wrote: > Hi, > > We're optimizing Cassandra right now for fairly random reads on a large > dataset. In this dataset, the values are

storing indexes on ssd

2018-02-10 Thread Dan Kinder
Hi, We're optimizing Cassandra right now for fairly random reads on a large dataset. In this dataset, the values are much larger than the keys. I was wondering, is it possible to have Cassandra write the *index* files (*-Index.db) to one drive (SSD), but write the *data* files (*-Data.db) to