thanks bunch stack for the response. agree, we should work towards it. very attractive feature for onlinestore use case of hbase
i will open a Jira and as we ramp-up on HBase here, might be able to discuss solutions and potentially work on adding this feature On Fri, Oct 1, 2010 at 8:31 AM, Stack <[email protected]> wrote: > Not currently. > > Blockcache is made of blocks pulled from HDFS. It'd be a little > awkward inserting hot records into the block cache w/o going via HDFS. > > But, yes, you have a good point that flush is disruptive of hot records. > > In the past we talked of a keyvalue cache on top of the block cache > but it fell out of favor because block cache seemed to be good enough > but sounds like we need to revive it or do some fancy dancing if > column family is marked in-memory, we keep around the snapshot of > memstore until we know the block cache has been populated? > > Any other suggestions? > > File an issue. We need to come up w/ a fix for this case. > > Thanks for writing the list, > St.Ack > > > > On Fri, Oct 1, 2010 at 1:26 AM, Abhijit Pol <[email protected]> wrote: > > we are trying to read efficiently a hot column family (in_memory=true, > > blockcaching=true) that get writes at say 500 qps and reads at 10,000 > qps. > > > > - as long as writes are in memstore we get them from memstore and its > fast > > - if we have read it once it will be at least in block cache (gets > priority > > due to in_memory=true) and subsequent reads are faster > > - however memstore flush puts records on disk which demands for disk IO > to > > get them back in block cache > > > > is there a way for memstore flush to go to blockcache? > > > > > > -- Abhijit > > >
