[
https://issues.apache.org/jira/browse/HBASE-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-1960:
----------------------------------
Description:
The master does not wait for DFS to come up in the circumstance where the DFS
master is started for the first time after format and no datanodes have been
started yet.
{noformat}
2009-11-07 11:47:28,115 INFO org.apache.hadoop.hbase.master.HMaster:
vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Sun Microsystems Inc.,
vmVersion=14.2-b01
2009-11-07 11:47:28,116 INFO org.apache.hadoop.hbase.master.HMaster:
vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError,
-XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
-Dhbase.log.dir=/mnt/hbase/logs,
-Dhbase.log.file=hbase-root-master-ip-10-242-15-159.log,
-Dhbase.home.dir=/usr/local/hbase-0.20.1/bin/.., -Dhbase.id.str=root,
-Dhbase.root.logger=INFO,DRFA,
-Djava.library.path=/usr/local/hbase-0.20.1/bin/../lib/native/Linux-amd64-64]
2009-11-07 11:47:28,247 INFO org.apache.hadoop.hbase.master.HMaster: My address
is ip-10-242-15-159.ec2.internal:60000
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer
Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/hbase/hbase.version could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
[...]
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery
for block null bad datanode[0] nodes == null
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Could not get
block locations. Source file "/hbase/hbase.version" - Aborting...
2009-11-07 11:47:28,729 FATAL org.apache.hadoop.hbase.master.HMaster: Not
starting HMaster because:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/hbase/hbase.version could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
{noformat}
Should probably sleep and retry the write a few times.
was:
The master does not wait for DFS to come up in the circumstance where the DFS
master is started for the first time after format and no datanodes have been
started yet.
{noformat}
2009-11-07 11:47:28,115 INFO org.apache.hadoop.hbase.master.HMaster:
vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Sun Microsystems Inc.,
vmVersion=14.2-b01
2009-11-07 11:47:28,116 INFO org.apache.hadoop.hbase.master.HMaster:
vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError,
-XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
-Dhbase.log.dir=/mnt/hbase/logs,
-Dhbase.log.file=hbase-root-master-ip-10-242-15-159.log,
-Dhbase.home.dir=/usr/local/hbase-0.20.1/bin/.., -Dhbase.id.str=root,
-Dhbase.root.logger=INFO,DRFA,
-Djava.library.path=/usr/local/hbase-0.20.1/bin/../lib/native/Linux-amd64-64]
2009-11-07 11:47:28,247 INFO org.apache.hadoop.hbase.master.HMaster: My address
is ip-10-242-15-159.ec2.internal:60000
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer
Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/hbase/hbase.version could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
[...]
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery
for block null bad datanode[0] nodes == null
2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Could not get
block locations. Source file "/hbase/hbase.version" - Aborting...
2009-11-07 11:47:28,729 FATAL org.apache.hadoop.hbase.master.HMaster: Not
starting HMaster because:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/hbase/hbase.version could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
{noformat}
Should probably sleep and retry the write a few times.
Fix Version/s: (was: 0.20.2)
0.20.3
> Master should wait for DFS to come up when creating hbase.version
> -----------------------------------------------------------------
>
> Key: HBASE-1960
> URL: https://issues.apache.org/jira/browse/HBASE-1960
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Priority: Minor
> Fix For: 0.21.0, 0.20.3
>
>
> The master does not wait for DFS to come up in the circumstance where the DFS
> master is started for the first time after format and no datanodes have been
> started yet.
> {noformat}
> 2009-11-07 11:47:28,115 INFO org.apache.hadoop.hbase.master.HMaster:
> vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Sun Microsystems Inc.,
> vmVersion=14.2-b01
> 2009-11-07 11:47:28,116 INFO org.apache.hadoop.hbase.master.HMaster:
> vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError,
> -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
> -Dhbase.log.dir=/mnt/hbase/logs,
> -Dhbase.log.file=hbase-root-master-ip-10-242-15-159.log,
> -Dhbase.home.dir=/usr/local/hbase-0.20.1/bin/.., -Dhbase.id.str=root,
> -Dhbase.root.logger=INFO,DRFA,
> -Djava.library.path=/usr/local/hbase-0.20.1/bin/../lib/native/Linux-amd64-64]
> 2009-11-07 11:47:28,247 INFO org.apache.hadoop.hbase.master.HMaster: My
> address is ip-10-242-15-159.ec2.internal:60000
> 2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer
> Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> /hbase/hbase.version could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
> [...]
> 2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery
> for block null bad datanode[0] nodes == null
> 2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Could not get
> block locations. Source file "/hbase/hbase.version" - Aborting...
> 2009-11-07 11:47:28,729 FATAL org.apache.hadoop.hbase.master.HMaster: Not
> starting HMaster because:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> /hbase/hbase.version could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
> {noformat}
> Should probably sleep and retry the write a few times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
