Hello everyone
I have a strange problem. The rocketMQ java client sometimes crashes
inexplicably.
Some JVM crash logs I think are critical as follows:
****************************************************************************
*********
--------------- T H R E A D ---------------
Current thread (0x0000fffef22d1000): JavaThread
"MQClientFactoryScheduledThread" [_thread_in_Java, id=3247460,
stack(0x0000fffdcaee0000,0x0000fffdcafe0000)]
siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr:
0x00000008e09fa8e5
Stack: [0x0000fffdcaee0000,0x0000fffdcafe0000], sp=0x0000fffdcafddce0,
free space=1015k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
j
io.netty.channel.nio.AbstractNioChannel.eventLoop()Lio/netty/channel/nio/Nio
EventLoop;+4
j
io.netty.channel.nio.AbstractNioChannel.eventLoop()Lio/netty/channel/EventLo
op;+1
j
io.netty.channel.AbstractChannelHandlerContext.executor()Lio/netty/util/conc
urrent/EventExecutor;+11
j
io.netty.channel.AbstractChannelHandlerContext.newPromise()Lio/netty/channel
/ChannelPromise;+9
j
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(Ljava/lang/Obje
ct;)Lio/netty/channel/ChannelFuture;+3
j
io.netty.channel.DefaultChannelPipeline.writeAndFlush(Ljava/lang/Object;)Lio
/netty/channel/ChannelFuture;+5
j
io.netty.channel.AbstractChannel.writeAndFlush(Ljava/lang/Object;)Lio/netty/
channel/ChannelFuture;+5
j
org.apache.rocketmq.remoting.netty.NettyRemotingAbstract.invokeSyncImpl(Lio/
netty/channel/Channel;Lorg/apache/rocketmq/remoting/protocol/RemotingCommand
;J)Lorg/apache/rocketmq/remoting/protocol/RemotingCommand;+48
J 17751 C2
org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(Ljava/lang
/String;Lorg/apache/rocketmq/remoting/protocol/RemotingCommand;J)Lorg/apache
/rocketmq/remoting/protocol/RemotingCommand; (195 bytes) @
0x0000fffeea65d2b0 [0x0000fffeea65d140+0x170]
J 19050 C2
org.apache.rocketmq.client.impl.factory.MQClientInstance.updateTopicRouteInf
oFromNameServer(Ljava/lang/String;ZLorg/apache/rocketmq/client/producer/Defa
ultMQProducer;)Z (673 bytes) @ 0x0000fffeeab01700 [0x0000fffeeab01480+0x280]
J 19490 C2
org.apache.rocketmq.client.impl.factory.MQClientInstance.updateTopicRouteInf
oFromNameServer()V (228 bytes) @ 0x0000fffeeac68368
[0x0000fffeeac666c0+0x1ca8]
J 20110 C2 org.apache.rocketmq.client.impl.factory.MQClientInstance$3.run()V
(27 bytes) @ 0x0000fffee7b2e738 [0x0000fffee7b2e700+0x38]
J 15811% C2
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/Thre
adPoolExecutor$Worker;)V (225 bytes) @ 0x0000fffee9390610
[0x0000fffee9390300+0x310]
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
V [libjvm.so+0x5dd6cc] JavaCalls::call_helper(JavaValue*, methodHandle*,
JavaCallArguments*, Thread*)+0xe3c
V [libjvm.so+0x5dae14] JavaCalls::call_virtual(JavaValue*, KlassHandle,
Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x28c
V [libjvm.so+0x5db3f4] JavaCalls::call_virtual(JavaValue*, Handle,
KlassHandle, Symbol*, Symbol*, Thread*)+0x4c
V [libjvm.so+0x648018] thread_entry(JavaThread*, Thread*)+0x68
V [libjvm.so+0x99f0c4] JavaThread::thread_main_inner()+0xcc
V [libjvm.so+0x99f464] JavaThread::run()+0x35c
V [libjvm.so+0x843c00] java_start(Thread*)+0x108
C [libpthread.so.0+0x88cc]
C [libc.so.6+0xda12c]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
heap address: 0x00000006c0000000, size: 4096 MB, Compressed Oops mode: Zero
based, Oop shift amount: 3
Narrow klass base: 0x0000000800000000, Narrow klass shift: 0
Compressed class space size: 1073741824 Address: 0x0000000800000000
Heap:
par new generation total 873856K, used 191380K [0x00000006c0000000,
0x0000000700000000, 0x0000000700000000)
eden space 699136K, 23% used [0x00000006c0000000, 0x00000006c9e4d658,
0x00000006eaac0000)
from space 174720K, 16% used [0x00000006f5560000, 0x00000006f71f7a10,
0x0000000700000000)
to space 174720K, 0% used [0x00000006eaac0000, 0x00000006eaac0000,
0x00000006f5560000)
concurrent mark-sweep generation total 1048576K, used 94096K
[0x0000000700000000, 0x0000000740000000, 0x00000007c0000000)
Metaspace used 131636K, capacity 138140K, committed 138304K, reserved
1171456K
class space used 15499K, capacity 16668K, committed 16704K, reserved
1048576K
Card table byte_map: [0x0000fffee6170000,0x0000fffee6980000] byte_map_base:
0x0000fffee2b70000
Marking Bits: (CMSBitMap*) 0x0000fffef0094bd8
Bits: [0x0000fffec4ff0000, 0x0000fffec7ff0000)
Mod Union Table: (CMSBitMap*) 0x0000fffef0094c98
Bits: [0x0000fffee5210000, 0x0000fffee52d0000)
Polling page: 0x0000fffef55c0000
CodeCache: size=131072Kb used=35285Kb max_used=57935Kb free=95787Kb
bounds [0x0000fffee77f0000, 0x0000fffeeb100000, 0x0000fffeef7f0000]
total_blobs=9048 nmethods=8111 adapters=850
compilation: enabled
Compilation events (10 events):
Event: 281183.616 Thread 0x0000fffef0163000 24927 4
org.springframework.scheduling.support.BitsCronField::nextOrSame (137 bytes)
Event: 281183.640 Thread 0x0000fffef0163000 nmethod 24927 0x0000fffee9bdccd0
code [0x0000fffee9bdd040, 0x0000fffee9bde910]
Event: 281869.069 Thread 0x0000fffef0169800 24928 3
io.netty.util.Recycler$DefaultHandle::recycle (27 bytes)
Event: 281869.070 Thread 0x0000fffef0169800 nmethod 24928 0x0000fffee80a2790
code [0x0000fffee80a2980, 0x0000fffee80a3140]
Event: 281869.174 Thread 0x0000fffef0169800 24929 3
io.netty.buffer.PoolThreadCache::allocate (45 bytes)
Event: 281869.174 Thread 0x0000fffef0169800 nmethod 24929 0x0000fffeea445910
code [0x0000fffeea445ac0, 0x0000fffeea445f18]
Event: 281898.618 Thread 0x0000fffef0169800 24930 3
io.netty.util.internal.shaded.org.jctools.queues.MpscChunkedArrayQueue::rela
xedOffer (6 bytes)
Event: 281898.618 Thread 0x0000fffef0169800 nmethod 24930 0x0000fffee9aefe90
code [0x0000fffee9af0000, 0x0000fffee9af01e8]
Event: 282678.618 Thread 0x0000fffef0167000 24931 4
io.netty.buffer.PoolThreadCache::allocate (45 bytes)
Event: 282678.634 Thread 0x0000fffef0167000 nmethod 24931 0x0000fffee8632ed0
code [0x0000fffee8633140, 0x0000fffee8633e18]
GC Heap History (10 events):
Event: 274419.423 GC heap before
{Heap before GC invocations=118 (full 3):
par new generation total 873856K, used 706451K [0x00000006c0000000,
0x0000000700000000, 0x0000000700000000)
eden space 699136K, 100% used [0x00000006c0000000, 0x00000006eaac0000,
0x00000006eaac0000)
from space 174720K, 4% used [0x00000006eaac0000, 0x00000006eb1e4de0,
0x00000006f5560000)
to space 174720K, 0% used [0x00000006f5560000, 0x00000006f5560000,
0x0000000700000000)
concurrent mark-sweep generation total 1048576K, used 94045K
[0x0000000700000000, 0x0000000740000000, 0x00000007c0000000)
Metaspace used 131616K, capacity 138140K, committed 138304K, reserved
1171456K
class space used 15499K, capacity 16668K, committed 16704K, reserved
1048576K
Event: 274419.433 GC heap after
Heap after GC invocations=119 (full 3):
par new generation total 873856K, used 3653K [0x00000006c0000000,
0x0000000700000000, 0x0000000700000000)
eden space 699136K, 0% used [0x00000006c0000000, 0x00000006c0000000,
0x00000006eaac0000)
from space 174720K, 2% used [0x00000006f5560000, 0x00000006f58f1538,
0x0000000700000000)
to space 174720K, 0% used [0x00000006eaac0000, 0x00000006eaac0000,
0x00000006f5560000)
concurrent mark-sweep generation total 1048576K, used 94069K
[0x0000000700000000, 0x0000000740000000, 0x00000007c0000000)
Metaspace used 131616K, capacity 138140K, committed 138304K, reserved
1171456K
class space used 15499K, capacity 16668K, committed 16704K, reserved
1048576K
}
Memory: 64k page, physical 15871808k(10249728k free), swap 0k(0k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.321-b07) for linux-aarch64
JRE (1.8.0_321-b07), built on Dec 15 2021 19:30:02 by "java_re" with gcc
7.3.0
time: Mon Nov 14 01:40:54 2022
timezone: CST
elapsed time: 282708.059187 seconds (3d 6h 31m 48s)
****************************************************************************
*********
It would be very grateful if someone could give some ideas when encountering
similar problems.
Thanks all.
*********************************************
--------------------------------------------------------------------
敬请留意：此电子邮件包含保密信息且仅限发送给预期的收件人。如果您不是此电子邮
件预期的收件人请立即通知发件人并从您的系统中删除此电子邮件。
本公司对于此电子邮件或其任何附件所携带的病毒造成的损失不承担任何责任。
PLEASE READ: The information contained in this e-mail is confidential and
intended for the named recipient(s) only. If you are not an intended
recipient of this e-mail please notify the sender immediately and delete
your copy from your system. Our company accepts no liability for any damage
sustained as a result of viruses in this e-mail or any attachment(s) to it.
