[
https://issues.apache.org/jira/browse/HBASE-19585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300583#comment-16300583
]
stack commented on HBASE-19585:
-------------------------------
[~apurtell] Thanks. Makes sense. Was looking at my native lib setup....
thinking it application. Let me play. Ugh... this is really old JVM on this
machine... let me fix. jdk1.8.0_73. Meantime here is a more complete stacktrace:
{code}
595664 java.lang.VerifyError: Bad type on operand stack
595665 Exception Details:
595666 Location:
595667
org/apache/hadoop/hdfs/protocolPB/PBHelperClient.convert(Lorg/apache/hadoop/hdfs/protocol/proto/HdfsProtos$ContentSummaryProto;)Lorg/apache/hadoop/fs/ContentSummary;
@ 98: invokestatic
595668 Reason:
595669 Type 'org/apache/hadoop/fs/ContentSummary$Builder' (current frame,
stack[1]) is not assignable to 'org/apache/hadoop/fs/QuotaUsage$Builder'
595670 Current Frame:
595671 bci: @98
595672 flags: { }
595673 locals: {
'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$ContentSummaryProto',
'org/apache/hadoop/fs/ContentSummary$Builder' }
595674 stack: {
'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$StorageTypeQuotaInfosProto',
'org/apache/hadoop/fs/ContentSummary$Builder' }
595675 Bytecode:
595676 0x0000000: 2ac7 0005 01b0 bb03 3159 b703 324c 2b2a
595677 0x0000010: b603 33b6 0334 2ab6 0335 b603 362a b603
595678 0x0000020: 37b6 0338 2ab6 0339 b603 3a2a b603 3bb6
595679 0x0000030: 033c 2ab6 033d b603 3e2a b603 3fb6 0340
595680 0x0000040: 2ab6 0341 b603 422a b603 43b6 0344 2ab6
595681 0x0000050: 0345 b603 4657 2ab6 0347 9900 0b2a b603
595682 0x0000060: 482b b803 492b b603 4ab0
595683 Stackmap Table:
595684 same_frame(@6)
595685 append_frame(@101,Object[#2126])
595686
595687 at java.lang.Class.forName0(Native Method)
595688 at java.lang.Class.forName(Class.java:264)
595689 at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createPBHelper(FanOutOneBlockAsyncDFSOutputHelper.java:414)
595690 at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.<clinit>(FanOutOneBlockAsyncDFSOutputHelper.java:551)
595691 at
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
595692 at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:159)
595693 at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:167)
595694 at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:99)
595695 at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:628)
595696 at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
595697 at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:760)
595698 at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:498)
595699 at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.<init>(AsyncFSWAL.java:264)
595700 at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createWAL(AsyncFSWALProvider.java:69)
595701 at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createWAL(AsyncFSWALProvider.java:44)
595702 at
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:139)
595703 at
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:55)
595704 at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:244)
595705 at
org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2122)
595706 at
org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:1315)
595707 at
org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:1196)
595708 at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1008)
595709 at java.lang.Thread.run(Thread.java:745)
{code}
> [WAL] "Unhandled: Bad type on operand stack" PBHelperClient.convert failing
> on HdfsProtos$ContentSummaryProto
> -------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-19585
> URL: https://issues.apache.org/jira/browse/HBASE-19585
> Project: HBase
> Issue Type: Bug
> Reporter: stack
>
> Testing, RS crashes soon after startup with below cryptic mess. This is the
> branch-2 started over a 0.98 data. [~Apache9] You have a clue sir?
> {code}
> 595126 2017-12-21 13:09:38,058 INFO
> [regionserver/ve0528.halxg.cloudera.com/10.17.240.22:16020]
> wal.AbstractFSWAL: WAL configuration: blocksize=128 MB, rollsize=121.60 MB,
> prefi x=ve0528.halxg.cloudera.com%2C16020%2C1513890565537, suffix=,
> logDir=hdfs://ve0524.halxg.cloudera.com:8020/hbase/WALs/ve0528.halxg.cloudera.com,16020,1513890565537,
> archiv eDir=hdfs://ve0524.halxg.cloudera.com:8020/hbase/oldWALs
> 595127 2017-12-21 13:09:38,107 ERROR
> [regionserver/ve0528.halxg.cloudera.com/10.17.240.22:16020]
> regionserver.HRegionServer: ***** ABORTING region server
> ve0528.halxg.cloudera.co m,16020,1513890565537: Unhandled: Bad type on
> operand stack
> 595128 Exception Details:
> 595129 Location:
> 595130
> org/apache/hadoop/hdfs/protocolPB/PBHelperClient.convert(Lorg/apache/hadoop/hdfs/protocol/proto/HdfsProtos$ContentSummaryProto;)Lorg/apache/hadoop/fs/ContentSummary;
> @ 98: invokestatic
> 595131 Reason:
> 595132 Type 'org/apache/hadoop/fs/ContentSummary$Builder' (current frame,
> stack[1]) is not assignable to 'org/apache/hadoop/fs/QuotaUsage$Builder'
> 595133 Current Frame:
> 595134 bci: @98
> 595135 flags: { }
> 595136 locals: {
> 'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$ContentSummaryProto',
> 'org/apache/hadoop/fs/ContentSummary$Builder' }
> 595137 stack: {
> 'org/apache/hadoop/hdfs/protocol/proto/HdfsProtos$StorageTypeQuotaInfosProto',
> 'org/apache/hadoop/fs/ContentSummary$Builder' }
> 595138 Bytecode:
> 595139 0x0000000: 2ac7 0005 01b0 bb03 3159 b703 324c 2b2a
> 595140 0x0000010: b603 33b6 0334 2ab6 0335 b603 362a b603
> 595141 0x0000020: 37b6 0338 2ab6 0339 b603 3a2a b603 3bb6
> 595142 0x0000030: 033c 2ab6 033d b603 3e2a b603 3fb6 0340
> 595143 0x0000040: 2ab6 0341 b603 422a b603 43b6 0344 2ab6
> 595144 0x0000050: 0345 b603 4657 2ab6 0347 9900 0b2a b603
> 595145 0x0000060: 482b b803 492b b603 4ab0
> 595146 Stackmap Table:
> 595147 same_frame(@6)
> 595148 append_frame(@101,Object[#2126])
> 595149 *****
> {code}
> 2.8.2 hadoop.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
