[
https://issues.apache.org/jira/browse/HBASE-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13954825#comment-13954825
]
Matt Corgan commented on HBASE-10771:
-------------------------------------
One of the problems with ByteBuffer is that it's difficult to copy multiple
bytes from BB to byte[] in a multi-threaded scenario, for example, when the
BlockCache is a bunch of DirectBBs and we want to have many simultaneous reader
threads reading from a hot block. For some reason, there's not a method to
copy a range of bytes from a random position in the DBB, so you have to either
acquire a lock on the DBB to take ownership of its position/limit fields, or
you have to "slice" it to give the reader a copy with its own position/limit.
Both locking and slicing would be very expensive when trying to assemble
encoded Cells in a tight loop.
One solution might be to have a pool of reader threads (i think we already do),
and to give each thread a pre-sliced BB for each big slab in the allocator.
The reader thread could reuse its private copy of the slab BB:
{code}
ByteRange dest = some ByteRange we want to copy into;
//the threadLocalBlock is a DirectByteBufferByteRange in this case.
//allocator sets position/limit to the block boundaries in the underlying slab.
//allocator manages thread-local pool of blocks and does ref counting.
ByteRange threadLocalBlock =
allocator.acquireReadOnlyThreadLocalBlock(blockCacheKey);
dest.copyFrom(threadLocalBlock, srcOffset, destOffset, length);
allocator.releaseReadOnlyThreadLocalBlock(blockCacheKey);
the array-backed ByteRange would do:
class ArrayByteRange implements ByteRange{
void copyFrom(ByteRange src, int srcOffset, int destOffset, int length){
if(src.hasArray()){
System.arraycopy(src.getArray(), srcOffset, this.array, destOffset,
length);
}else{
DirectByteBuffer srcBuffer = ((DirectByteBuffer)src).getBuffer();
int previousPosition = srcBuffer.position();
int previousLimit = srcBuffer.limit();
srcBuffer.position(srcOffset);
srcBuffer.limit(srcOffset + length);
srcBuffer.get(this.array);//copies the bytes
srcBuffer.position(previousPosition);
srcBuffer.limit(previousLimit);
}
}
}
while the DirectByteBufferByteRange would do:
class DirectByteBufferByteRange implements ByteRange{
void copyFrom(ByteRange src, int srcOffset, int destOffset, int length){
if(src.hasArray()){
byte[] srcArray = ((ArrayByteBuffer)src).getArray();
this.buffer.put(srcArray, srcOffset, length);//copies the bytes
}else{
DirectByteBuffer srcBuffer = ((DirectByteBuffer)src).getBuffer();
int previousPosition = srcBuffer.position();
int previousLimit = srcBuffer.limit();
srcBuffer.position(srcOffset);
srcBuffer.limit(srcOffset + length);
this.buffer.put(srcBuffer);//copies the bytes
srcBuffer.position(previousPosition);
srcBuffer.limit(previousLimit);
}
}
}
{code}
Need to be careful with offset/limit calculations. They may be more complex
than in that example.
> Primitive type put/get APIs in ByteRange
> -----------------------------------------
>
> Key: HBASE-10771
> URL: https://issues.apache.org/jira/browse/HBASE-10771
> Project: HBase
> Issue Type: Improvement
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.99.0
>
> Attachments: HBASE-10771.patch, HBASE-10771_V2.patch
>
>
> While doing HBASE-10713 I came across the need to write int/long (and read
> also) from a ByteRange. CellBlocks are backed by ByteRange. So we can add
> such APIs.
> Also as per HBASE-10750 we return a ByteRange from MSLAB and also discussion
> under HBASE-10191 suggest we can have BR backed HFileBlocks etc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
