HBase master does not start
---------------------------
Key: HBASE-5513
URL: https://issues.apache.org/jira/browse/HBASE-5513
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.4
Environment: SUSE Linux Enterprise Server 11 64-bit Service Pack 1
Reporter: Quan Liu
Priority: Blocker
Used Cloudera Manager 3.7.3 to install Cloudera CDH3U3 on a cluster of EC2
instances. After installation, HBase master cannot start. Both HDFS NN and DN
have started successfully.
The HBase log shows
PM INFO org.apache.hadoop.hbase.metrics
MetricsString added: url
Mar 2, 4:41:21 PM INFO org.apache.hadoop.hbase.metrics
MetricsString added: version
Mar 2, 4:41:21 PM INFO org.apache.hadoop.hbase.metrics
new MBeanInfo
Mar 2, 4:41:21 PM INFO org.apache.hadoop.hbase.metrics
new MBeanInfo
Mar 2, 4:41:21 PM INFO
org.apache.hadoop.hbase.master.metrics.MasterMetrics
Initialized
Mar 2, 4:41:21 PM INFO
org.apache.hadoop.hbase.master.ActiveMasterManager
Master=epoch-node-101:60000
Mar 2, 4:41:22 PM WARN org.apache.hadoop.hdfs.DFSClient
DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
java.io.IOException: File /hbase/hbase.version could only be replicated to 0
nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1520)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:665)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
at org.apache.hadoop.ipc.Client.call(Client.java:1107)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
at $Proxy6.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy6.addBlock(Unknown Source)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3553)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3421)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2100(DFSClient.java:2627)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2822)
Mar 2, 4:41:22 PM WARN org.apache.hadoop.hdfs.DFSClient
Error Recovery for block null bad datanode[0] nodes == null
Mar 2, 4:41:22 PM WARN org.apache.hadoop.hdfs.DFSClient
Could not get block locations. Source file "/hbase/hbase.version" - Aborting...
Mar 2, 4:41:22 PM WARN org.apache.hadoop.hbase.util.FSUtils
Unable to create version file at hdfs://epoch-node-101:8020/hbase, retrying:
java.io.IOException: File /hbase/hbase.version could only be replicated to 0
nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1520)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:665)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
