[
https://issues.apache.org/jira/browse/HBASE-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268435#comment-13268435
]
Bob Copeland commented on HBASE-4956:
-------------------------------------
With regard to readFully() being everywhere -- perhaps it would be less
intrusive to subclass DataInput, or to modify one of the read functions further
down in the stack like org.apache.hadoop.net.SocketInputStream.read() to do the
chunking. You guys know the code better than I.
These numbers are for standalone mode. I can't easily do a test on our
distributed cluster but I don't think it would change the numbers much. I'll
upload my test script and then anyone who wants can repeat the experiment on
their setup.
Each trial consists of a run of "hbase org.jruby.Main thread_get.rb 1", which
does 100 gets of a ~10 MB row, 3 times in a row, computing avg & stddev of
wall-clock time. Maxread variable is changed for every run.
{code}
size avg stddev
~~~~ ~~~~~~ ~~~~~~
1 33.690 .10
1k 3.072 .18
2k 3.395 .19
4k 3.493 .16
8k 3.306 .19
16k 2.918 .21
32k 2.835 .06
64k 2.689 .08
128k 3.048 .03
len 3.328 .08
{code}
> Control direct memory buffer consumption by HBaseClient
> -------------------------------------------------------
>
> Key: HBASE-4956
> URL: https://issues.apache.org/jira/browse/HBASE-4956
> Project: HBase
> Issue Type: New Feature
> Reporter: Ted Yu
>
> As Jonathan explained here
> https://groups.google.com/group/asynchbase/browse_thread/thread/c45bc7ba788b2357?pli=1
> , standard hbase client inadvertently consumes large amount of direct memory.
> We should consider using netty for NIO-related tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
