Hi,
We use hbase 0.20.3
The server running master ran out of disk space.
I cleaned up some space:
/dev/mapper/VolGroup00-LogVol00
303346048 268141776 19795120 94% /
But I still cannot stop hbase - non-stop dot printing:
[sjc1-qa-certiq1.sjc1:hadoop 26345]stop-hbase.sh
stopping master........
Here is from master log:
2010-05-19 03:53:44,033 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode /hbase/master
got 10.201.8.247:60000
2010-05-19 03:53:44,034 INFO
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: getMaster
attempt 9 of 10 failed; no more retrying.
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:308)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:831)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:712)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:489)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:465)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:512)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:341)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1258)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1282)
2010-05-19 03:53:44,034 ERROR org.apache.hadoop.hbase.master.HMaster: master
is not running
2010-05-19 03:53:44,035 INFO org.apache.zookeeper.ZooKeeper: Closing
session: 0x128ad17448a007d
2010-05-19 03:53:44,035 INFO org.apache.zookeeper.ClientCnxn: Closing
ClientCnxn for session: 0x128ad17448a007d
2010-05-19 03:53:44,039 INFO org.apache.zookeeper.ClientCnxn: Exception
while closing send thread for session 0x128ad17448a007d : Read error rc = -1
java.nio.DirectByteBuffer[pos=0 lim=4 cap=4]
2010-05-19 03:53:44,144 INFO org.apache.zookeeper.ClientCnxn: Disconnecting
ClientCnxn for session: 0x128ad17448a007d
2010-05-19 03:53:44,144 INFO org.apache.zookeeper.ZooKeeper: Session:
0x128ad17448a007d closed
2010-05-19 03:53:44,144 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Closed connection with
ZooKeeper
202,1-4 99%
Here is jps output:
[sjc1-qa-certiq1.sjc1:hadoop 26341]jps
2287 Jps
23365 JobTracker
23561 HMaster
Has anyone seen similar problem ?
Thanks
