[
https://issues.apache.org/jira/browse/HBASE-27187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ji Shen updated HBASE-27187:
----------------------------
Environment:
Node1: HBASE 2.3.5 Hadoop3.2.2 zookeeper-3.4.5
Node2: HBASE 2.3.5 Hadoop3.2.2 zookeeper-3.4.5
Node3: HBASE 2.3.5 Hadoop3.2.2 zookeeper-3.4.5
was:
HBASE 2.3.5
Hadoop3.2.2
> master.HMaster: Failed to become active master , after reboot the hosts
> -----------------------------------------------------------------------
>
> Key: HBASE-27187
> URL: https://issues.apache.org/jira/browse/HBASE-27187
> Project: HBase
> Issue Type: Bug
> Environment: Node1: HBASE 2.3.5 Hadoop3.2.2 zookeeper-3.4.5
> Node2: HBASE 2.3.5 Hadoop3.2.2 zookeeper-3.4.5
> Node3: HBASE 2.3.5 Hadoop3.2.2 zookeeper-3.4.5
> Reporter: Ji Shen
> Priority: Major
>
> 2022-07-09 20:17:02,408 INFO
> [ReadOnlyZKClient-hadoop01:2181,hadoop02:2181,hadoop03:2181@0x23c8e44a-SendThread(hadoop01:2181)]
> zookeeper.ClientCnxn: Socket connection established, initiating session,
> client: /172.19.21.101:57094, server: hadoop01/172.19.21.101:2181
> 2022-07-09 20:17:02,409 INFO
> [ReadOnlyZKClient-hadoop01:2181,hadoop02:2181,hadoop03:2181@0x23c8e44a-SendThread(hadoop01:2181)]
> zookeeper.ClientCnxn: Session establishment complete on server
> hadoop01/172.19.21.101:2181, sessionid = 0xff81e2d459f50248, negotiated
> timeout = 40000
> 2022-07-09 20:17:02,434 INFO [master/hadoop01:16000:becomeActiveMaster]
> region.MasterRegion: Create or load local region for table 'master:store',
> \{NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS =>
> '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE',
> COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}ZIZIaUaUaU
> 2022-07-09 20:17:02,437 INFO [master/hadoop01:16000:becomeActiveMaster]
> region.MasterRegionFlusherAndCompactor: Injected flushSize=134217728,
> flushPerChanges=1000000, flushIntervalMs=900000
> 2022-07-09 20:17:02,486 INFO [master/hadoop01:16000:becomeActiveMaster]
> wal.WALFactory: Instantiating WALProvider of type class
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider
> 2022-07-09 20:17:02,522 ERROR [master/hadoop01:16000:becomeActiveMaster]
> master.HMaster: Failed to become active master
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:222)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:309)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:948)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2240)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
> at java.lang.Thread.run(Thread.java:748)
> 2022-07-09 20:17:02,523 ERROR [master/hadoop01:16000:becomeActiveMaster]
> master.HMaster: ***** ABORTING master hadoop01,16000,1657369019206: Unhandled
> exception. Starting shutdown. *****
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:222)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:309)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:948)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2240)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
> at java.lang.Thread.run(Thread.java:748)
> 2022-07-09 20:17:02,523 INFO [master/hadoop01:16000:becomeActiveMaster]
> regionserver.HRegionServer: ***** STOPPING region server
> 'hadoop01,16000,1657369019206' *****
> 2022-07-09 20:17:02,523 INFO [master/hadoop01:16000:becomeActiveMaster]
> regionserver.HRegionServer: STOPPED: Stopped by
> master/hadoop01:16000:becomeActiveMaster
> 2022-07-09 20:17:02,523 INFO [master/hadoop01:16000]
> regionserver.HRegionServer: Stopping infoServer
> 2022-07-09 20:17:02,529 INFO [master/hadoop01:16000] handler.ContextHandler:
> Stopped
> o.e.j.w.WebAppContext@648d0e6d\{/,null,UNAVAILABLE}{file:/home/hadoop/app/hbase-2.3.5/hbase-webapps/master}pOpPpP
> 2022-07-09 20:17:02,534 INFO [master/hadoop01:16000]
> server.AbstractConnector: Stopped
> ServerConnector@512d6e60\{HTTP/1.1,[http/1.1]}qPqPqP\{0.0.0.0:16010}xI
> 2022-07-09 20:17:02,534 INFO [master/hadoop01:16000] handler.ContextHandler:
> Stopped
> o.e.j.s.ServletContextHandler@a66e580\{/static,file:///home/hadoop/app/hbase-2.3.5/hbase-webapps/static/,UNAVAILABLE}
> 2022-07-09 20:17:02,534 INFO [master/hadoop01:16000] handler.ContextHandler:
> Stopped
> o.e.j.s.ServletContextHandler@352c44a8\{/logs,file:///home/hadoop/app/hbase-2.3.5/logs/,UNAVAILABLE}
> 2022-07-09 20:17:02,535 INFO [master/hadoop01:16000]
> regionserver.HRegionServer: aborting server hadoop01,16000,1657369019206
> 2022-07-09 20:17:02,535 INFO [master/hadoop01:16000]
> regionserver.HRegionServer: stopping server hadoop01,16000,1657369019206; all
> regions closed.
> 2022-07-09 20:17:02,535 INFO [master/hadoop01:16000] hbase.ChoreService:
> Chore service for: master/hadoop01:16000 had [] on shutdown
> 2022-07-09 20:17:02,536 WARN [master/hadoop01:16000]
> master.ActiveMasterManager: Failed get of master address:
> java.io.IOException: Can't get master address from ZooKeeper; znode data ==
> null
> 2022-07-09 20:17:02,537 INFO [master/hadoop01:16000] hbase.ChoreService:
> Chore service for: hadoop01:16000.splitLogManager. had [ScheduledChore
> name=SplitLogManager Timeout Monitor, period=1000, unit=MILLISECONDS] on
> shutdown
> 2022-07-09 20:17:02,537 INFO [master/hadoop01:16000] ipc.NettyRpcServer:
> Stopping server on /172.19.21.101:16000
> 2022-07-09 20:17:02,638 INFO
> [ReadOnlyZKClient-hadoop01:2181,hadoop02:2181,hadoop03:2181@0x23c8e44a]
> zookeeper.ZooKeeper: Session: 0xff81e2d459f50248 closed
> 2022-07-09 20:17:02,638 INFO
> [ReadOnlyZKClient-hadoop01:2181,hadoop02:2181,hadoop03:2181@0x23c8e44a-EventThread]
> zookeeper.ClientCnxn: EventThread shut down for session: 0xff81e2d459f50248
> 2022-07-09 20:17:02,737 INFO [master/hadoop01:16000] zookeeper.ZooKeeper:
> Session: 0xff81e2d459f50246 closed
> 2022-07-09 20:17:02,737 INFO [main-EventThread] zookeeper.ClientCnxn:
> EventThread shut down for session: 0xff81e2d459f50246
> 2022-07-09 20:17:02,737 INFO [master/hadoop01:16000]
> regionserver.HRegionServer: Exiting; stopping=hadoop01,16000,1657369019206;
> zookeeper connection closed.
> 2022-07-09 20:17:02,738 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: HMaster Aborted
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:261)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:149)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3085)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
