[
https://issues.apache.org/jira/browse/HBASE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855457#comment-16855457
]
Zheng Hu commented on HBASE-22532:
----------------------------------
Our non-data ( bloom_chunk or leaf_index ) block were > 128KB, because we have
the default settting in HFileBlockIndex.java:
{code}
static final int DEFAULT_MAX_CHUNK_SIZE = 128 * 1024;
/**
* The maximum size guideline for index blocks (both leaf, intermediate, and
* root). If not specified, <code>DEFAULT_MAX_CHUNK_SIZE</code> is used.
*/
public static final String MAX_CHUNK_SIZE_KEY = "hfile.index.block.max.size";
{code}
So seems it's OK here, not a bug. That's to say if someone disabled the block
cache in clusters, then should use a larger
hbase.ipc.server.allocator.buffer.size, such as 130KB, otherwise we may
encoutered the cpu-costing checksum validation, which may limit the Get/Scan
throughtput.
> There's still too much cpu wasting on validating checksum even if
> buffer.size=65KB
> ----------------------------------------------------------------------------------
>
> Key: HBASE-22532
> URL: https://issues.apache.org/jira/browse/HBASE-22532
> Project: HBase
> Issue Type: Sub-task
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Major
> Attachments: async-prof-pid-27827-cpu-3.svg,
> async-prof-pid-64695-cpu-1.svg
>
>
> After disabled the block cache, and with the following config:
> {code}
> # Disable the block cache
> hfile.block.cache.size=0
> hbase.ipc.server.allocator.buffer.size=66560
> hbase.ipc.server.reservoir.minimal.allocating.size=0
> {code}
> The ByteBuff for block should be expected to be a SingleByteBuff, which will
> use the hadoop native lib to validate the checksum, while in the cpu flame
> graph
> [async-prof-pid-27827-cpu-3.svg|https://issues.apache.org/jira/secure/attachment/12970683/async-prof-pid-27827-cpu-3.svg],
> we can still see that about 32% CPU wasted on PureJavaCrc32#update, which
> means it's not using the faster hadoop native lib.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
