gaofeng created HBASE-27452:
-------------------------------
Summary: RegionServer JVM crash when scan
Key: HBASE-27452
URL: https://issues.apache.org/jira/browse/HBASE-27452
Project: HBase
Issue Type: Task
Components: scan
Affects Versions: 2.3.5
Reporter: gaofeng
Attachments: hs_err_pid117788.log
In our cluster, we found region servers may be crashed in several cases
hs_err_pid117788.log
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fdc733a9a4e, pid=117788, tid=0x00007fdc0a34b700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_171-b11) (build
1.8.0_171-b11)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.171-b11 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# j org.apache.hadoop.hbase.ipc.NettyServerRpcConnection.isConnectionOpen()Z+4
#
# Failed to write core dump. Core dumps have been disabled. To enable core
dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x00007fd48004f000): JavaThread
"RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020" daemon
[_thread_in_Java, id=118639, stack(0x00007fdc0a24b000,0x00007fdc0a34c000)]
regionserver日志
2022-10-28 03:33:03,611 WARN
[RpcServer.default.FPBQ.Fifo.handler=25,queue=0,port=16020] ipc.RpcServer:
(responseTooSlow): {"call":"Scan(org.apache.hadoop.h
base.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1666899153611","responsesize":"19","method":"Scan","param":"scanner_id:
119667009731
85298703 number_of_rows: 1000 close_scanner: false next_call_seq: 0
client_handles_partials: true client_handles_heartbeats: t
<TRUNCATED>","processingtimems
":30000,"client":"10.64.143.22:48550","queuetimems":0,"class":"HRegionServer","scandetails":"table:
action_user_track_v2 region: action_user_track_v2,\\xE8\\
x00\\x00\\x00-\\xAA\\xF9\\xFB\\x00\\x00\\x01\{\\xCF\\xA6\\xDE\\x09,1645585735023.efb249fd8f4c26d4911d50f14fc30ce8."}
2022-10-28 03:33:03,611 WARN
[RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020]
hfile.BlockCacheUtil: Caching an already cached block: 58a6823d3958
49099bb226e18acb79b2_SeqId_6__1032126997. This is harmless and can happen in
rare cases (see HBASE-8547)
2022-10-28 03:33:03,611 WARN
[RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020]
hfile.BlockCacheUtil: Caching an already cached block: eb1c30882099
45f6b5006db45d5fe286_SeqId_4__1032126997. This is harmless and can happen in
rare cases (see HBASE-8547)
2022-10-28 03:33:03,612 WARN
[RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020]
hfile.BlockCacheUtil: Caching an already cached block: eb1c30882099
45f6b5006db45d5fe286_SeqId_4__1032149468. This is harmless and can happen in
rare cases (see HBASE-8547)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
