[jira] [Commented] (HBASE-14202) Reduce garbage we create
[
https://issues.apache.org/jira/browse/HBASE-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681946#comment-14681946
]
Hadoop QA commented on HBASE-14202:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749758/HBASE-14202.patch
against master branch at commit 7d4de20cafd6b765bd5f33df72fc0e630d1731f7.
ATTACHMENT ID: 12749758
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15042//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15042//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15042//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15042//console
This message is automatically generated.
Reduce garbage we create
Key: HBASE-14202
URL: https://issues.apache.org/jira/browse/HBASE-14202
Project: HBase
Issue Type: Sub-task
Components: regionserver, Scanners
Affects Versions: 2.0.0
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0
Attachments: HBASE-14202.patch
2 optimizations wrt no# short living objects we create
1. IOEngine#read call to read from L2 cache is always creating a Pair object
to return the BB and MemoryType. We can avoid this by making the read API to
return a Cacheable. Pass the CacheableDeserializer, to be used, also to the
read API. Setter for MemoryType is already there in Cacheable interface.
2. ByteBuff#asSubByteBuffer(int, int, Pair) avoids Pair object creation
every time as we pass the shared Pair object. Still as pair can take only
Objects, the primitive int has to be boxed into an Integer object every time.
This can be avoided by creating a new Pair type which is a pair of an Object
and a primitive int.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14202) Reduce garbage we create
[ https://issues.apache.org/jira/browse/HBASE-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682409#comment-14682409 ] Hudson commented on HBASE-14202: FAILURE: Integrated in HBase-TRUNK #6714 (See [https://builds.apache.org/job/HBase-TRUNK/6714/]) HBASE-14202 Reduce garbage we create. (anoopsamjohn: rev d49b1f60f4cec6f04768f28f678bf4329bfa7364) * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java * hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java * hbase-common/src/main/java/org/apache/hadoop/hbase/nio/MultiByteBuff.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/ByteBufferIOEngine.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/IOEngine.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ObjectIntPair.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestByteBufferIOEngine.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestFileIOEngine.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestBufferedDataBlockEncoder.java * hbase-common/src/main/java/org/apache/hadoop/hbase/nio/SingleByteBuff.java * hbase-common/src/test/java/org/apache/hadoop/hbase/nio/TestMultiByteBuff.java Reduce garbage we create Key: HBASE-14202 URL: https://issues.apache.org/jira/browse/HBASE-14202 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Affects Versions: 2.0.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0 Attachments: HBASE-14202.patch 2 optimizations wrt no# short living objects we create 1. IOEngine#read call to read from L2 cache is always creating a Pair object to return the BB and MemoryType. We can avoid this by making the read API to return a Cacheable. Pass the CacheableDeserializer, to be used, also to the read API. Setter for MemoryType is already there in Cacheable interface. 2. ByteBuff#asSubByteBuffer(int, int, Pair) avoids Pair object creation every time as we pass the shared Pair object. Still as pair can take only Objects, the primitive int has to be boxed into an Integer object every time. This can be avoided by creating a new Pair type which is a pair of an Object and a primitive int. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14202) Reduce garbage we create
[ https://issues.apache.org/jira/browse/HBASE-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681223#comment-14681223 ] Anoop Sam John commented on HBASE-14202: Thanks Stack. Will commit later tonight once QA passes. bq.The Object/int pair is ugly Agree.. Bit odd it is.. But it helps us a lot wrt garbage we create. :-) This kind of more fine tuning sub tasks will come in under this Jira. We are doing more profiling stuff wrt CPU time and object/memory usage. Reduce garbage we create Key: HBASE-14202 URL: https://issues.apache.org/jira/browse/HBASE-14202 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Affects Versions: 2.0.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0 Attachments: HBASE-14202.patch 2 optimizations wrt no# short living objects we create 1. IOEngine#read call to read from L2 cache is always creating a Pair object to return the BB and MemoryType. We can avoid this by making the read API to return a Cacheable. Pass the CacheableDeserializer, to be used, also to the read API. Setter for MemoryType is already there in Cacheable interface. 2. ByteBuff#asSubByteBuffer(int, int, Pair) avoids Pair object creation every time as we pass the shared Pair object. Still as pair can take only Objects, the primitive int has to be boxed into an Integer object every time. This can be avoided by creating a new Pair type which is a pair of an Object and a primitive int. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14202) Reduce garbage we create
[ https://issues.apache.org/jira/browse/HBASE-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681209#comment-14681209 ] stack commented on HBASE-14202: --- The return of Cacheable is a nice one. The Object/int pair is ugly but confined and +1 on less object creation. +1 if hadoopqa says it good. Reduce garbage we create Key: HBASE-14202 URL: https://issues.apache.org/jira/browse/HBASE-14202 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Affects Versions: 2.0.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0 Attachments: HBASE-14202.patch 2 optimizations wrt no# short living objects we create 1. IOEngine#read call to read from L2 cache is always creating a Pair object to return the BB and MemoryType. We can avoid this by making the read API to return a Cacheable. Pass the CacheableDeserializer, to be used, also to the read API. Setter for MemoryType is already there in Cacheable interface. 2. ByteBuff#asSubByteBuffer(int, int, Pair) avoids Pair object creation every time as we pass the shared Pair object. Still as pair can take only Objects, the primitive int has to be boxed into an Integer object every time. This can be avoided by creating a new Pair type which is a pair of an Object and a primitive int. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
