[
https://issues.apache.org/jira/browse/HBASE-7239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634562#comment-13634562
]
Devaraj Das commented on HBASE-7239:
------------------------------------
AFAICT, the problem is there in the current codebase. In the current code, we
read cells as opposed to whole Result objects (as in 0.94). However, we read
all the cells in a response as a block.
Here is where the problem seems to be present (in the method
HBaseClient.Connection.readResponse):
{noformat}
if (responseHeader.hasCellBlockMeta()) {
int size = responseHeader.getCellBlockMeta().getLength();
byte [] cellBlock = new byte[size];
IOUtils.readFully(this.in, cellBlock, 0, cellBlock.length);
cellBlockScanner = ipcUtil.createCellScanner(this.codec,
this.compressor, cellBlock);
}
{noformat}
The readFully call is the suspect.
> Verify protobuf serialization is correctly chunking upon read to avoid direct
> memory OOMs
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-7239
> URL: https://issues.apache.org/jira/browse/HBASE-7239
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Priority: Critical
> Fix For: 0.95.1
>
>
> Result.readFields() used to read from the input stream in 8k chunks to avoid
> OOM issues with direct memory.
> (Reading variable sized chunks into direct memory prevent the JVM from
> reusing the allocated direct memory and direct memory is only collected
> during full GCs)
> This is just to verify protobufs parseFrom type methods do the right thing as
> well so that we do not reintroduce this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
