[
https://issues.apache.org/jira/browse/HBASE-11351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031775#comment-14031775
]
Lars Hofhansl commented on HBASE-11351:
---------------------------------------
Next:
* Make it work in 0.96+ (protobufs). Although could keep the current
serialization as well
* Better serialization (that makes use of the the sorted nature of the keys and
does prefix compression)
* Only store key portion in the key of the map
* Fork the set implementation so that we do not need to store KV in a map
(which in offheap means two copies of the KV)
* Do comparisons off heap! Needs more access into the BTree details (i.e. to
the data before it is deserialized onto the heap again)
Not sure how much time I'll get on those. If somebody wants to experiment I'm
happy to collaborate.
> Experiments with OffHeap memstores
> ----------------------------------
>
> Key: HBASE-11351
> URL: https://issues.apache.org/jira/browse/HBASE-11351
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Attachments: 11351-0.94-v2.txt, 11351-0.94.txt
>
>
> I have been experimenting with offheap memstores using an offheap BTree
> implementation of ConcurrentMap provided by MapDB (www.mapdb.org)
> So far I hooked this up in an experimental way with inefficient serialization
> (in the sense that due to our special usage of the set api we need to back
> the set by a map, which in the off heap case means we need to serialize and
> store the value twice. It is used as key and value). But it works.
> It's quite a bit slower and due to need to deserialize upon scan it produces
> *more* short term garbage.
> Let's use this issue to discuss.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
