[hbase] branch HBASE-21879 updated: HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator (#279)

[openinx]

This is an automated email from the ASF dual-hosted git repository.

openinx pushed a commit to branch HBASE-21879
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/HBASE-21879 by this push:
     new e2c8e65  HBASE-22483 It's better to use 65KB as the default buffer 
size in ByteBuffAllocator (#279)
e2c8e65 is described below

commit e2c8e65cdfe5dde6f0a2336d48f104439a327aad
Author: openinx <open...@gmail.com>
AuthorDate: Fri May 31 17:53:00 2019 +0800

    HBASE-22483 It's better to use 65KB as the default buffer size in 
ByteBuffAllocator (#279)
---
 .../apache/hadoop/hbase/io/ByteBuffAllocator.java    | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git 
a/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java 
b/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java
index 75a4699..c85675b 100644
--- 
a/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java
+++ 
b/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java
@@ -71,8 +71,24 @@ public class ByteBuffAllocator {
   public static final String MAX_BUFFER_COUNT_KEY = 
"hbase.ipc.server.allocator.max.buffer.count";
 
   public static final String BUFFER_SIZE_KEY = 
"hbase.ipc.server.allocator.buffer.size";
-  // 64 KB. Making it same as the chunk size what we will write/read to/from 
the socket channel.
-  public static final int DEFAULT_BUFFER_SIZE = 64 * 1024;
+
+  /**
+   * There're some reasons why better to choose 65KB(rather than 64KB) as the 
default buffer size:
+   * <p>
+   * 1. Almost all of the data blocks have the block size: 64KB + delta, whose 
delta is very small,
+   * depends on the size of lastKeyValue. If we set buffer.size=64KB, then 
each block will be
+   * allocated as a MultiByteBuff: one 64KB DirectByteBuffer and delta bytes 
HeapByteBuffer, the
+   * HeapByteBuffer will increase the GC pressure. Ideally, we should let the 
data block to be
+   * allocated as a SingleByteBuff, it has simpler data structure, faster 
access speed, less heap
+   * usage.
+   * <p>
+   * 2. Since the blocks are MultiByteBuff when using buffer.size=64KB, so we 
have to calculate the
+   * checksum by an temp heap copying (see HBASE-21917), while if it's a 
SingleByteBuff, we can
+   * speed the checksum by calling the hadoop' checksum in native lib, which 
is more faster.
+   * <p>
+   * For performance comparison, please see HBASE-22483.
+   */
+  public static final int DEFAULT_BUFFER_SIZE = 65 * 1024;
 
   public static final String MIN_ALLOCATE_SIZE_KEY =
       "hbase.ipc.server.reservoir.minimal.allocating.size";