[
https://issues.apache.org/jira/browse/HBASE-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204296#comment-13204296
]
Todd Lipcon commented on HBASE-5355:
------------------------------------
bq. Todd in your proposal above we'd convert KVs to your new form before
putting payload on wire and then undo it on other side?
Yep, something like that. But hopefully it's just reference twiddling and not
any actual data copying.
bq. Long time back Ryan tried to a custom compression before putting stuff on
the wire and then undoing it on other end hoping it would help some w/ latency
but he found it upped latency
I remember that too, but if I remember correctly it was going through the whole
codec stream interface, which necessitates extra copies, etc. There's a lot of
room for optimization there by adding new APIs to the hadoop codec stuff that
operate on direct byte buffers in-place. And doing the "application level
compression" as described above should be faster as well.
> Compressed RPC's for HBase
> --------------------------
>
> Key: HBASE-5355
> URL: https://issues.apache.org/jira/browse/HBASE-5355
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.89.20100924
> Reporter: Karthik Ranganathan
> Assignee: Karthik Ranganathan
>
> Some application need ability to do large batched writes and reads from a
> remote MR cluster. These eventually get bottlenecked on the network. These
> results are also pretty compressible sometimes.
> The aim here is to add the ability to do compressed calls to the server on
> both the send and receive paths.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
