[
https://issues.apache.org/jira/browse/HBASE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705457#action_12705457
]
Jonathan Gray commented on HBASE-794:
-------------------------------------
There are two instances where it would be helpful to have an RPC call that we
could access the received byte[] directly.
Sending of PUTS and receiving of GETS. This is being changed in 880 to being
List<KeyValue> (does not require serializiation/deserialization). I believe,
by far, the biggest improvement we could make to help with our GC woes during
imports is doing this for writes.
Just thought we should keep this in mind as we explore rpc stuff.
Right now, we allocate data for writes *at least 3 times* on the way in to the
memcache. The last two times, it is at the individual keyval level (lots of
small allocations). Using KeyValue, we could read the entire buffer into a
single byte[] and pass around KeyValue pointers all the way to the Memcache.
These would be grouped by row in the case of batching.
I do not think we need to go down the path of pooling. Changes like above, and
moving away from Trees wherever possible as we're working on now, should help
tremendously. If we do end up needing more improvements, the optimizations
would be isolated to the data structure backing Memcache. I think it'd be a
fun project... but i agree that it can bite you hard with a GC and we should
avoid it at all costs (short of rewrites in c) :)
> Language neutral IPC as a first class component of HBase architecture
> ---------------------------------------------------------------------
>
> Key: HBASE-794
> URL: https://issues.apache.org/jira/browse/HBASE-794
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, master, regionserver
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Minor
>
> This issue considers making a language neutral IPC mechanism and wire format
> a first class component of HBase architecture. Clients could talk to the
> master and regionserver using this protocol instead of HRPC at their option.
> Options for language neutral IPC include:
> * Thrift: http://incubator.apache.org/thrift/
> * Protocol buffers: http://code.google.com/p/protobuf/
> * XDR: http://en.wikipedia.org/wiki/External_Data_Representation
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
