[
https://issues.apache.org/jira/browse/HBASE-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218218#comment-15218218
]
stack commented on HBASE-15509:
-------------------------------
What is the mysterious 'len'?
84 * @param len - length
HFileContext is meant to be immutable. You add methods that do get/set.
We add new configs for that non-existent 'expert hbase user' to twiddle:
"hbase.hfile.write.buffer-reuse.max.size" The on/off switch is fine but this
new 'sizing' with no means of measuring if an arbitrary setting is improving
things or making them worse.
What is TLS in writeBufferTLS? Transport Layer Security ?
What is expected lifecycle of the thread local byte buffer? If "Thread local
buffer which is used to compress and encode file block before writing it out to
output.", then why are we keeping ti around?
You introduce white space #1027 and unorthodox bracketing of methods -- see
canReuseBufferForCurrentBlock
In other than a synthetic loading, the below looks like it could make things
worse. We will have churn as size fluctuates and we have the added cost of
get/set a thread local:
1016 if (canReuseBufferForCurrentBlock()) {
1017 byte[] currentBuffer = writeBufferTLS.get();
1018 if (currentBuffer == null || currentBuffer.length <
baosInMemory.size()) {
1019 currentBuffer = new byte[baosInMemory.size()];
1020 writeBufferTLS.set(currentBuffer);
1021 }
Seeing your reset of BBOS to length zero, and considering all the BB work that
has been going on in here abouts and elsewhere to do caching, wouldn't use of
BBs be more natural? More natural than adding methods to BBOS?
How effective is this caching? Block has to be same size or less than incoming
one. How often will this be the case? Any idea? How do we mitigate case where a
Cell is 1G? All my threads now have thread locals of 1G though I've gone back
to serving Cells of 16k?
As per [~ram_krish], any perf numbers? Any GC graphs? This on a laptop with a
fixed loading?
> Avoid copy of block data in HFileBlock$Writer.finishBlock
> ---------------------------------------------------------
>
> Key: HBASE-15509
> URL: https://issues.apache.org/jira/browse/HBASE-15509
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Attachments: HBASE-15509-v1.patch, HBASE-15509-v2.patch
>
>
> This calls ByteArrayOutputStream.toByteArray() which creates a copy of data
> block.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
