Yeah, I stumbled onto that as well. Both our implementations are very similar in context, we both use DirectByteBuffers, and copy the stored data out of the heap. I haven't looked at his code too much, but it seems that he optimizes for a more general case, while my allocator exploits the fact that the cached objects are very similar in size. He creates his own slice method and pointer classes, for instance, in order to save a few bytes. I don't bother, as the average cached object in HBase is far larger.
Additionally, I used the memcached allocation model, where we divide a slab into equally sized blocks, and simply waste a bit of memory in order to deal with fragmentation. This works extremely well, as HBase block sizes are generally very close to eachother. >From glancing through his code, it seems he allows for fragmented writes, and has a move function, quite possibly for compaction. I haven't reviewed the code thoroughly though. My code is available in hbase trunk right now - so if you'd like to do a review/comparison, thats available to you. Also, I'd guess the final difference is that mine has already been integrated into hbase ;). If theres any improvement that can be made to 4027, feel free to let me know - it would be greatly appreciated. ~Li On Mon, Sep 5, 2011 at 1:34 PM, Otis Gospodnetic <[email protected] > wrote: > Hi Li, > > I stumbled upon https://github.com/raffaeleguidi/DirectMemory today and > then noticed your slides in this email below. > Could you please compare the 2 approaches/implementations? > > Thanks, > Otis > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Hadoop - HBase > Hadoop ecosystem search :: http://search-hadoop.com/ > > > ----- Original Message ----- > > From: Li Pi <[email protected]> > > To: [email protected]; [email protected] > > Cc: > > Sent: Tuesday, August 23, 2011 5:03 PM > > Subject: HBase User Group Meeting - Off Heap Caching Slides: > > > > Not sure which email list this goes to, but somebody requested these. > > > > https://docs.google.com/present/view?id=d23xkzr_55hgnvngf6 > > >
