[
https://issues.apache.org/jira/browse/HBASE-20673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004385#comment-17004385
]
zhuobin zheng commented on HBASE-20673:
---------------------------------------
I Can't add attachment. So I add some info in this comment.
version: cdh5-1.2.0_5.14.4
-----------------------------------------------------------------------------------------------------------------------------------------------------
Profiler Info:
ns percent samples top
---------- ------- ------- ---
219079399723 16.81% 21956 itable stub
187885376151 14.42% 18830 itable stub
168718141522 12.95% 16909 itable stub
149243475899 11.45% 14957 itable stub
108522505239 8.33% 10876 itable stub
54090604368 4.15% 5421 itable stub
45659351417 3.50% 4576 org.apache.hadoop.hbase.CellComparator.compareRows_[j]
41398429360 3.18% 4149 itable stub
32259010132 2.48% 3233 itable stub
30253160585 2.32% 3032 itable stub
10897498052 0.84% 1092 HeapRegion::block_size(HeapWord const*) const
10467104761 0.80% 1049 org.apache.hadoop.hbase.KeyValue.getFamilyLength_[j]
10176340086 0.78% 1020 G1ParScanThreadState::trim_queue()
9867105912 0.76% 989 G1ParScanThreadState::copy_to_survivor_space(InCSetState,
oopDesc*, markOopDesc*)
9768378849 0.75% 979 itable stub
-----------------------------------------------------------------------------------------------------------------------------------------------------
Jstack Info:
"RpcServer.RW.fifo.Q.write.handler=73,queue=1,port=60020" #133 daemon prio=5
os_prio=0 tid=0x00007faad0097800 nid=0x403d runnable [0x00007faaceeec000]
java.lang.Thread.State: RUNNABLE
at org.apache.hadoop.hbase.CellComparator.compareRows(CellComparator.java:186)
at org.apache.hadoop.hbase.CellComparator.compare(CellComparator.java:63)
at org.apache.hadoop.hbase.KeyValue$KVComparator.compare(KeyValue.java:2020)
at org.apache.hadoop.hbase.KeyValue$KVComparator.compare(KeyValue.java:1897)
at
java.util.concurrent.ConcurrentSkipListMap.cpr(ConcurrentSkipListMap.java:655)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:899)
at
java.util.concurrent.ConcurrentSkipListMap.put(ConcurrentSkipListMap.java:1581)
at
org.apache.hadoop.hbase.regionserver.CellSkipListSet.add(CellSkipListSet.java:134)
at
org.apache.hadoop.hbase.regionserver.DefaultMemStore.addToCellSet(DefaultMemStore.java:242)
at
org.apache.hadoop.hbase.regionserver.DefaultMemStore.internalAdd(DefaultMemStore.java:276)
at
org.apache.hadoop.hbase.regionserver.DefaultMemStore.add(DefaultMemStore.java:233)
at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:686)
at
org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3807)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3280)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2944)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2886)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:765)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:716)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2146)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2191)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:183)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:163)
"RpcServer.RW.fifo.Q.write.handler=71,queue=8,port=60020" #131 daemon prio=5
os_prio=0 tid=0x00007faad0093800 nid=0x403b runnable [0x00007faacf0ee000]
java.lang.Thread.State: RUNNABLE
at
org.apache.hadoop.hbase.CellComparator.compareColumns(CellComparator.java:157)
at
org.apache.hadoop.hbase.CellComparator.compareWithoutRow(CellComparator.java:224)
at org.apache.hadoop.hbase.CellComparator.compare(CellComparator.java:66)
at org.apache.hadoop.hbase.KeyValue$KVComparator.compare(KeyValue.java:2020)
at org.apache.hadoop.hbase.KeyValue$KVComparator.compare(KeyValue.java:1897)
at
java.util.concurrent.ConcurrentSkipListMap.cpr(ConcurrentSkipListMap.java:655)
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:899)
at
java.util.concurrent.ConcurrentSkipListMap.put(ConcurrentSkipListMap.java:1581)
at
org.apache.hadoop.hbase.regionserver.CellSkipListSet.add(CellSkipListSet.java:134)
at
org.apache.hadoop.hbase.regionserver.DefaultMemStore.addToCellSet(DefaultMemStore.java:242)
at
org.apache.hadoop.hbase.regionserver.DefaultMemStore.internalAdd(DefaultMemStore.java:276)
at
org.apache.hadoop.hbase.regionserver.DefaultMemStore.add(DefaultMemStore.java:233)
at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:686)
at
org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3807)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3280)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2944)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2886)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:765)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:716)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2146)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2191)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:183)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:163)
-----------------------------------------------------------------------------------------------------------------------------------------------------
KeyValue Type Analyze:
Env: All Cell no tag.
Read Situation:
NUM Compare Left Compare Right
22 Compare: org.apache.hadoop.hbase.KeyValue
org.apache.hadoop.hbase.NoTagsKeyValue
211 Compare: org.apache.hadoop.hbase.KeyValue
org.apache.hadoop.hbase.KeyValue$KeyOnlyKeyValue
116743 Compare: org.apache.hadoop.hbase.NoTagsKeyValue
org.apache.hadoop.hbase.NoTagsKeyValue
Write Situation:
NUM Compare Left Compare Right
90597 Compare: org.apache.hadoop.hbase.KeyValue org.apache.hadoop.hbase.KeyValue
Situation Analyze:
Why Read almost NoTagsKeyValue?
https://github.com/apache/hbase/blob/branch-1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java#L206
Why Read Has Some KeyValue, or KeyOnlyKeyValue?
some Block Seek. check cell. will create this instance in code.
Why Write all KeyValue Instance?
https://github.com/apache/hbase/blob/branch-1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java#L311
-----------------------------------------------------------------------------------------------------------------------------------------------------
KeyValue implements too many interface.
public class KeyValue implements Cell, HeapSize, Cloneable, SettableSequenceId,
SettableTimestamp {
-----------------------------------------------------------------------------------------------------------------------------------------------------
And I try to change write KeyValue to NoTag, So in the situation All Cell no
tag. almost compare cell is NoTagKeyValue
> Reduce the number of Cell implementations; the profusion is distracting to
> users and JIT
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-20673
> URL: https://issues.apache.org/jira/browse/HBASE-20673
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
> Reporter: Michael Stack
> Assignee: Michael Stack
> Priority: Major
> Attachments: 0001-current.patch, 0001-current.patch, hits.20673.png
>
>
> We have a wild blossom of Cell implementations in hbase. Purge the bulk of
> them. Make it so we do one type well. JIT gets confused if it has an abstract
> or an interface and then the instantiated classes are myriad (megamorphic).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
