Right, I think one naive approach would be to take CSLM, convert it to use AtomicIntegerArray instead of objects (basically emulate objects in blocks of ints), and of course not implement delete functionality.
The first challenge is making an AtomicIntegerArray based linked list where the value is pointed to by an int into a byte block data structure. The next is a 2nd AIA that emulates the internal 'Index' class of CSLM. Thoughts? On Tue, Jun 14, 2011 at 3:51 PM, Stack <[email protected]> wrote: > Or do as Todd suggested a while back and pull in CSLM and hack it to > suit our needs; its public domain. > St.Ack > > On Tue, Jun 14, 2011 at 3:23 PM, Jason Rutherglen > <[email protected]> wrote: >>> See this J: >>> http://www.cloudera.com/blog/2011/03/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-3/ >> >> Interesting. >> >> In Lucene for realtime search we have the same problem where the >> best/only option right now for the realtime terms dictionary is to use >> ConcurrentSkipListMap. However the Node, key, value, and Index >> objects all are pointers that consume RAM and need to be cleaned up. >> >> Lucene right now during indexing does not generate much object pointer >> garbage. We have the same problem in the two projects. >> >> Maybe there's a way to create an AtomicIntegerArray based system that >> uses int pointers to perform the same function as a the concurrent >> skip list. It would greatly reduce garbage generation. >> >> On Tue, Jun 14, 2011 at 10:14 AM, Stack <[email protected]> wrote: >>> See this J: >>> http://www.cloudera.com/blog/2011/03/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-3/ >>> St.Ack >>> >>> On Mon, Jun 13, 2011 at 10:58 PM, Jason Rutherglen >>> <[email protected]> wrote: >>>> Is there an issue with garbage generated from using CSLM in the MemStore? >>>> >>> >> >
