OK I'll work on the change later because there's another problem to solve: the overhead for cache is too big that 1.4mil records (1k each) consumed all of the 6gb memory of JVM (I guess 4gb are consumed by the row cache). I'm thinking that ConcurrentHashMap is not a good choice for LRU and the row cache needs to store compressed key data to reduce memory usage. I'll do more investigation on this and let you know.
-Weijun On Tue, Feb 16, 2010 at 9:22 PM, Jonathan Ellis <jbel...@gmail.com> wrote: > ... tell you what, if you write the option-processing part in > DatabaseDescriptor I will do the actual cache part. :) > > On Tue, Feb 16, 2010 at 11:07 PM, Jonathan Ellis <jbel...@gmail.com> > wrote: > > https://issues.apache.org/jira/secure/CreateIssue!default.jspa<https://issues.apache.org/jira/secure/CreateIssue%21default.jspa>, > but > > this is pretty low priority for me. > > > > On Tue, Feb 16, 2010 at 8:37 PM, Weijun Li <weiju...@gmail.com> wrote: > >> Just tried to make quick change to enable it but it didn't work out :-( > >> > >> ColumnFamily cachedRow = > cfs.getRawCachedRow(mutation.key()); > >> > >> // What I modified > >> if( cachedRow == null ) { > >> cfs.cacheRow(mutation.key()); > >> cachedRow = cfs.getRawCachedRow(mutation.key()); > >> } > >> > >> if (cachedRow != null) > >> cachedRow.addAll(columnFamily); > >> > >> How can I open a ticket for you to make the change (enable row cache > write > >> through with an option)? > >> > >> Thanks, > >> -Weijun > >> > >> On Tue, Feb 16, 2010 at 5:20 PM, Jonathan Ellis <jbel...@gmail.com> > wrote: > >>> > >>> On Tue, Feb 16, 2010 at 7:17 PM, Jonathan Ellis <jbel...@gmail.com> > wrote: > >>> > On Tue, Feb 16, 2010 at 7:11 PM, Weijun Li <weiju...@gmail.com> > wrote: > >>> >> Just started to play with the row cache feature in trunk: it seems > to > >>> >> be > >>> >> working fine so far except that for RowsCached parameter you need to > >>> >> specify > >>> >> number of rows rather than a percentage (e.g., "20%" doesn't work). > >>> > > >>> > 20% works, but it's 20% of the rows at server startup. So on a fresh > >>> > start that is zero. > >>> > > >>> > Maybe we should just get rid of the % feature... > >>> > >>> (Actually, it shouldn't be hard to update this on flush, if you want > >>> to open a ticket.) > >> > >> > > >
