You might also consider moving back down to 7u79
> On Feb 15, 2016, at 10:35 AM, Jonathan Leech <[email protected]> wrote: > > Has anyone else seen this? Happening under load in jdk 1.7.0_80 / phoenix > 4.5.2 - cloudera labs. Based on the source code, It seems the JVM is calling > the wrong toObject(), and then dumping. The correct toObject() method is a > couple parent classes away with some generics and Sun / Oracle must have > gotten something wrong with that special case under load. Thinking I may be > able to just patch in a call to the correct method to stop the bleeding. > Anyone else have other ideas, thoughts, or also want the patch? > > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > code) > > J 7432 C2 > org.apache.phoenix.schema.types.PDecimal.toObject([BIILorg/apache/phoenix/schema/types/PDataType;Lorg/apache/phoenix/schema/SortOrder;Ljava/lang/Integer;Ljava/lang/Integer;)Ljava/lang/Object; > (376 bytes) @ 0x00007f8332f3de52 [0x00007f8332f3d9c0+0x492] > > J 14641 C2 > org.apache.phoenix.expression.DecimalAddExpression.evaluate(Lorg/apache/phoenix/schema/tuple/Tuple;Lorg/apache/hadoop/hbase/io/ImmutableBytesWritable;)Z > (178 bytes) @ 0x00007f83349ecde8 [0x00007f83349eca00+0x3e8] > > J 11096 C2 > org.apache.phoenix.expression.CoerceExpression.evaluate(Lorg/apache/phoenix/schema/tuple/Tuple;Lorg/apache/hadoop/hbase/io/ImmutableBytesWritable;)Z > (68 bytes) @ 0x00007f8333ed824c [0x00007f8333ed81c0+0x8c] > > J 9742 C2 > org.apache.phoenix.compile.ExpressionProjector.getValue(Lorg/apache/phoenix/schema/tuple/Tuple;Lorg/apache/phoenix/schema/types/PDataType;Lorg/apache/hadoop/hbase/io/ImmutableBytesWritable;)Ljava/lang/Object; > (88 bytes) @ 0x00007f83329f57b8 [0x00007f83329f5760+0x58] > > J 8783 C2 > org.apache.phoenix.compile.UpsertCompiler.upsertSelect(Lorg/apache/phoenix/compile/StatementContext;Lorg/apache/phoenix/schema/TableRef;Lorg/apache/phoenix/compile/RowProjector;Lorg/apache/phoenix/iterate/ResultIterator;[I[I)Lorg/apache/phoenix/execute/MutationState; > (633 bytes) @ 0x00007f833367c448 [0x00007f833367ba00+0xa48] > > J 10636 C2 > org.apache.phoenix.compile.MutatingParallelIteratorFactory.newIterator(Lorg/apache/phoenix/compile/StatementContext;Lorg/apache/phoenix/iterate/ResultIterator;Lorg/apache/hadoop/hbase/client/Scan;Ljava/lang/String;)Lorg/apache/phoenix/iterate/PeekingResultIterator; > (168 bytes) @ 0x00007f8333d63924 [0x00007f8333d63180+0x7a4] > > J 8646 C2 > org.apache.phoenix.iterate.ParallelIterators$1.call()Ljava/lang/Object; (5 > bytes) @ 0x00007f833358010c [0x00007f833357fe40+0x2cc] > > J 9466 C2 org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run()V > (12 bytes) @ 0x00007f83338e7240 [0x00007f83338e7180+0xc0] > > J 7047 C2 > java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V > (225 bytes) @ 0x00007f8332e4971c [0x00007f8332e49580+0x19c] > > J 12402 C2 java.util.concurrent.ThreadPoolExecutor$Worker.run()V (58 bytes) @ > 0x00007f833420ad2c [0x00007f833420ac80+0xac] > > j java.lang.Thread.run()V+26 > > v ~StubRoutines::call_stub > > V [libjvm.so+0x6020a5] JavaCalls::call_helper(JavaValue*, methodHandle*, > JavaCallArguments*, Thread*)+0x365 > > V [libjvm.so+0x600b08] JavaCalls::call(JavaValue*, methodHandle, > JavaCallArguments*, Thread*)+0x28 > > V [libjvm.so+0x600dd7] JavaCalls::call_virtual(JavaValue*, KlassHandle, > Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x197 > > V [libjvm.so+0x600ef7] JavaCalls::call_virtual(JavaValue*, Handle, > KlassHandle, Symbol*, Symbol*, Thread*)+0x47 > > V [libjvm.so+0x67f8d5] thread_entry(JavaThread*, Thread*)+0xe5 > > V [libjvm.so+0x95bfff] JavaThread::thread_main_inner()+0xdf > > V [libjvm.so+0x95c105] JavaThread::run()+0xf5 > > V [libjvm.so+0x8238c8] java_start(Thread*)+0x108
