[
https://issues.apache.org/jira/browse/HBASE-22134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807077#comment-16807077
]
Todd Lipcon commented on HBASE-22134:
-------------------------------------
Yea, I think I meant -XX:+PrintCompilation. Also -XX:+TraceDeoptimization or
maybe -XX:+DebugDeoptimization would be useful for this case. Not entirely
certain which one was spouting the above.
Apparently the folks at FB have hit this issue in the context of Presto.
Discussing with them here:
https://groups.google.com/d/topic/presto-users/RCVd_UVMW5I/discussion
bq. This probably happens all over the codebase I'm guessing
Not sure of that -- when this happens the performance degradation is pretty
dramatic. I think someone would probably have spotted it.
> JIT deoptimization in Cell.write
> --------------------------------
>
> Key: HBASE-22134
> URL: https://issues.apache.org/jira/browse/HBASE-22134
> Project: HBase
> Issue Type: Bug
> Components: Performance
> Affects Versions: 2.0.2
> Reporter: Todd Lipcon
> Priority: Major
>
> I was looking at a profile of a workload which was running compaction very
> slowly, and saw that the top CPU consumers were from JVM internals regarding
> deoptimization. I managed to write a little systemtap script to extract the
> deoptimization log and got the following in a tight loop:
> "Uncommon trap: trap_request=0xffffff67 fr.pc=0x00007f85bcdb8644"
> "Uncommon trap: reason=unstable_if action=none pc=0x00007f85bcdb8644
> method=org.apache.hadoop.hbase.io.encoding.NoneEncoder.write(Lorg/apache/hadoop/hbase/Cell;)I
> @ 67"
> "DEOPT PACKING pc=0x00007f85bcdb8644 sp=0x00007f84d3d83080"
> "DEOPT UNPACKING pc=0x00007f85b5005229 sp=0x00007f84d3d82f30 mode 2"
> The java stack is spending most of its time at:
> java.lang.Thread.State: RUNNABLE
> at
> org.apache.hadoop.hbase.io.encoding.NoneEncoder.write(NoneEncoder.java:57)
> at
> org.apache.hadoop.hbase.io.hfile.NoOpDataBlockEncoder.encode(NoOpDataBlockEncoder.java:55)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.write(HFileBlock.java:983)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.append(HFileWriterImpl.java:740)
> This was with Oracle JDK 1.8.0_112. Likely a JDK bug but perhaps some
> reorganization of this code path could help avoid triggering the bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
