[
https://issues.apache.org/jira/browse/HBASE-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024115#comment-17024115
]
Pankaj Kumar commented on HBASE-23739:
--------------------------------------
Following exception occurred during WAL split at region server side,
{code}
2020-01-27T06:44:38.059Z, RpcRetryingCaller{globalStartTime=1580107459823,
pause=100, maxAttempts=9}, org.apache.hadoop.hbase.PleaseHoldException:
org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2835)
at
org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1071)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3005)
at
org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:548)
at
org.apache.hadoop.hbase.client.HBaseAdmin.getDescriptor(HBaseAdmin.java:363)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.configContextForNonMetaWriter(BoundedRecoveredHFilesOutputSink.java:220)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.createRecoveredHFileWriter(BoundedRecoveredHFilesOutputSink.java:211)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.append(BoundedRecoveredHFilesOutputSink.java:115)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$writeRemainingEntryBuffers$3(BoundedRecoveredHFilesOutputSink.java:153)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
{code}
Finally HMaster abort with timedout,
{code}
2020-01-27 12:18:17,821 ERROR
[master/root1-ThinkPad-T440p:16000:becomeActiveMaster] master.HMaster: Failed
to become active master
java.lang.IllegalStateException: Expected the service ClusterSchemaServiceImpl
[FAILED] to be RUNNING, but the service has FAILED
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:366)
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:312)
at
org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1293)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1074)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2139)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Timedout 300000ms waiting for namespace table
to be assigned and enabled: tableName=hbase:namespace, state=ENABLED
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:108)
at
org.apache.hadoop.hbase.master.ClusterSchemaServiceImpl.doStart(ClusterSchemaServiceImpl.java:63)
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:248)
at
org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1291)
... 4 more
2020-01-27 12:18:17,822 ERROR
[master/root1-ThinkPad-T440p:16000:becomeActiveMaster] master.HMaster: Master
server abort: loaded coprocessors are:
[org.apache.hadoop.hbase.security.access.AccessController]
2020-01-27 12:18:17,822 ERROR
[master/root1-ThinkPad-T440p:16000:becomeActiveMaster] master.HMaster: *****
ABORTING master root1-thinkpad-t440p,16000,1580107380446: Unhandled exception.
Starting shutdown. *****
java.lang.IllegalStateException: Expected the service ClusterSchemaServiceImpl
[FAILED] to be RUNNING, but the service has FAILED
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:366)
at
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:312)
at
org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1293)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1074)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2139)
{code}
> BoundedRecoveredHFilesOutputSink should read the table descriptor directly
> --------------------------------------------------------------------------
>
> Key: HBASE-23739
> URL: https://issues.apache.org/jira/browse/HBASE-23739
> Project: HBase
> Issue Type: Sub-task
> Reporter: Guanghao Zhang
> Priority: Major
>
> Read from meta or filesystem?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
