Hi, We've had the multiple instances where a 16 node grid completely crashes with the following error.
we use all our caches OFF HEAP caches in our configuration. We are running Ignite 1.7 and running Java 1.8u60-b27 on Linux Has anyone seen a similar issue? Thanks Yohan I've attached a few of the hs_err log files. Please let us know if you need anymore information. # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007ff13f1c2524, pid=3402054, tid=140670445545216 # # JRE version: Java(TM) SE Runtime Environment (8.0_60-b27) (build 1.8.0_60-b27) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode linux-amd64 compressed oops) # Problematic frame: # v ~StubRoutines::jint_disjoint_arraycopy # # Core dump written. Default location: /prod/Java/apps/data-grid-server-ignite/2.2.31/bin/core or core.3402054 # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x00007ff1552f9800): JavaThread "pub-#12%DataGridServer-Production%" [_thread_in_Java, id=3402569, stack(0x00007ff063dfe000,0x00007ff063eff000)] siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr: 0x000000076a800000 Stack: [0x00007ff063dfe000,0x00007ff063eff000], sp=0x00007ff063efc490, free space=1017k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) v ~StubRoutines::jint_disjoint_arraycopy J 6572 C2 org.apache.ignite.internal.util.GridUnsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V (14 bytes) @ 0x00007ff13f52f1a8 [0x00007ff13f52f180+0x28] j org.apache.ignite.internal.util.io.GridUnsafeDataInput.readShortArray()[S+88 j org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readShortArray()[S+4 J 25716 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride()Ljava/lang/Object; (883 bytes) @ 0x00007ff1437bbc4c [0x00007ff1437ba2a0+0x19ac] J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124 bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c] J 12272 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readFields(Ljava/lang/Object;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8] J 12508 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readSerializable(Ljava/lang/Class;Ljava/util/List;Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object; (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4] J 16693 C2 org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.read(Lorg/apache/ignite/marshaller/optimized/OptimizedObjectInputStream;)Ljava/lang/Object; (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8] J 25716 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride()Ljava/lang/Object; (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0] J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124 bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c] J 12272 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readFields(Ljava/lang/Object;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8] J 12508 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readSerializable(Ljava/lang/Class;Ljava/util/List;Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object; (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4] J 16693 C2 org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.read(Lorg/apache/ignite/marshaller/optimized/OptimizedObjectInputStream;)Ljava/lang/Object; (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8] J 25716 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride()Ljava/lang/Object; (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0] J 6560 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readArrayList()Ljava/util/ArrayList; (47 bytes) @ 0x00007ff140621050 [0x00007ff140620c80+0x3d0] J 25716 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride()Ljava/lang/Object; (883 bytes) @ 0x00007ff1437bbbac [0x00007ff1437ba2a0+0x190c] J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124 bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c] J 12272 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readFields(Ljava/lang/Object;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8] J 12508 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readSerializable(Ljava/lang/Class;Ljava/util/List;Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object; (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4] J 16693 C2 org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.read(Lorg/apache/ignite/marshaller/optimized/OptimizedObjectInputStream;)Ljava/lang/Object; (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8] J 25716 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride()Ljava/lang/Object; (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0] J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124 bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c] J 12272 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readFields(Ljava/lang/Object;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$ClassFields;)V (336 bytes) @ 0x00007ff140833d78 [0x00007ff140833980+0x3f8] J 12508 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readSerializable(Ljava/lang/Class;Ljava/util/List;Ljava/lang/reflect/Method;Lorg/apache/ignite/marshaller/optimized/OptimizedClassDescriptor$Fields;)Ljava/lang/Object; (176 bytes) @ 0x00007ff14178ee74 [0x00007ff14178e780+0x6f4] J 16693 C2 org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.read(Lorg/apache/ignite/marshaller/optimized/OptimizedObjectInputStream;)Ljava/lang/Object; (130 bytes) @ 0x00007ff140804868 [0x00007ff1408044a0+0x3c8] J 25716 C2 org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride()Ljava/lang/Object; (883 bytes) @ 0x00007ff1437bbe80 [0x00007ff1437ba2a0+0x1be0] J 6523 C2 java.io.ObjectInputStream.readObject()Ljava/lang/Object; (124 bytes) @ 0x00007ff13f4e0d9c [0x00007ff13f4e0760+0x63c] J 25841 C2 org.apache.ignite.marshaller.optimized.OptimizedMarshaller.unmarshal([BLjava/lang/ClassLoader;)Ljava/lang/Object; (148 bytes) @ 0x00007ff140c27724 [0x00007ff140c27300+0x424] J 25109 C1 org.apache.ignite.internal.processors.job.GridJobWorker.initialize(Lorg/apache/ignite/internal/managers/deployment/GridDeployment;Ljava/lang/Class;)Z (295 bytes) @ 0x00007ff141e6a83c [0x00007ff141e6a500+0x33c] J 24444 C1 org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(Lorg/apache/ignite/cluster/ClusterNode;Lorg/apache/ignite/internal/GridJobExecuteRequest;)V (1342 bytes) @ 0x00007ff1434879ac [0x00007ff143481280+0x672c] J 24443 C1 org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(Ljava/util/UUID;Ljava/lang/Object;)V (147 bytes) @ 0x00007ff140d08ab4 [0x00007ff140d075a0+0x1514] J 14472 C2 org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(Lorg/apache/ignite/internal/managers/communication/GridIoMessage;Ljava/util/UUID;)V (55 bytes) @ 0x00007ff141ddc8e4 [0x00007ff141ddc5e0+0x304] J 11641 C2 org.apache.ignite.internal.managers.communication.GridIoManager$5.run()V (52 bytes) @ 0x00007ff1414b02cc [0x00007ff1414b0240+0x8c] j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95 j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+19 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub V [libjvm.so+0x68bbe6] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x1056 V [libjvm.so+0x68c0f1] JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x321 V [libjvm.so+0x68c597] JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*)+0x47 V [libjvm.so+0x7232d0] thread_entry(JavaThread*, Thread*)+0xa0 V [libjvm.so+0xa68f3f] JavaThread::thread_main_inner()+0xdf V [libjvm.so+0xa6906c] JavaThread::run()+0x11c V [libjvm.so+0x91cb88] java_start(Thread*)+0x108 C [libpthread.so.0+0x7aa1] hs_err_pid3410800.log <http://apache-ignite-users.70518.x6.nabble.com/file/n8356/hs_err_pid3410800.log> hs_err_pid3410096.log <http://apache-ignite-users.70518.x6.nabble.com/file/n8356/hs_err_pid3410096.log> hs_err_pid3408462.log <http://apache-ignite-users.70518.x6.nabble.com/file/n8356/hs_err_pid3408462.log> -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/JVM-Crash-SIGSEGV-on-GridUnsafe-copyMemory-tp8356.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
