[
https://issues.apache.org/jira/browse/HBASE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850394#comment-16850394
]
binlijin commented on HBASE-22489:
----------------------------------
When regionserver stopping, throw wrong max sequence id also aborting
regionserver
{code}
Tue May 28 23:06:31 CST 2019 Terminating regionserver
2019-05-28 23:06:31,873 INFO [Thread-5] regionserver.ShutdownHook: Shutdown
hook starting; hbase.shutdown.hook=true;
fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@26d96e5
2019-05-28 23:06:31,873 INFO [Thread-5] regionserver.HRegionServer: *****
STOPPING region server '10.11.45.152,16020,1559046718578' *****
2019-05-28 23:06:31,873 INFO [Thread-5] regionserver.HRegionServer: STOPPED:
Shutdown hook
2019-05-28 23:06:31,950 ERROR
[RS_CLOSE_REGION-regionserver/hbaseperf-4:16020-6] regionserver.HRegionServer:
***** ABORTING region server 10.11.45.152,16020,1559046718578: Unrecoverable
exception while closing region
user,a4432394ed8-dca3-4eab-a61f-86541787b600155854,1559023636867.770e1f69047d51d90cc24a6b5927458d.,
still finishing close *****
java.io.IOException: The new max sequence id 9187005 is less than the old max
sequence id 9187006
at
org.apache.hadoop.hbase.wal.WALSplitter.writeRegionSequenceIdFile(WALSplitter.java:684)
at
org.apache.hadoop.hbase.regionserver.HRegion.writeRegionCloseMarker(HRegion.java:1177)
at
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1709)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1526)
at
org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:110)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
> When regionServer stopping open region cause server aborting
> ------------------------------------------------------------
>
> Key: HBASE-22489
> URL: https://issues.apache.org/jira/browse/HBASE-22489
> Project: HBase
> Issue Type: Bug
> Reporter: binlijin
> Priority: Major
>
> When regionserver stopping, region open will ABORTING regionserver, need to
> take care it.
> {code}
> 2019-05-28 20:32:48,672 WARN
> [RS_OPEN_REGION-regionserver/hbaseperf-8:16020-15]
> handler.AssignRegionHandler: Fatal error occurred while opening region
> user,f7777768,1559033479968.6f1508a4480447698cf8b04cd850698f., aborting...
> org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server
> 10.11.45.52,16020,1559046489201 stopping
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1505)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2206)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:139)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-05-28 20:32:48,677 ERROR
> [RS_OPEN_REGION-regionserver/hbaseperf-8:16020-15]
> regionserver.HRegionServer: ***** ABORTING region server
> 10.11.45.52,16020,1559046489201: Failed to open region
> user,f7777768,1559033479968.6f1508a4480447698cf8b04cd850698f. and can not
> recover *****
> org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server
> 10.11.45.52,16020,1559046489201 stopping
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1505)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2206)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:139)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-05-28 20:32:48,677 ERROR
> [RS_OPEN_REGION-regionserver/hbaseperf-8:16020-15]
> regionserver.HRegionServer: RegionServer abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
