Hello, I need some help with my issue,
We have been running HBase on Amazon EMR (v 0.92) for quite some months,
today we had a serious problem, Hbase was throwing some "connection
refused" errors in out client applications, after some investigation, we
found out that one of the two EC2 instances running hbase (the
slave/regionserver one) was disk full (800+ G), we can connect to both
of the instances, but when we try to run a command from the HBase Shell
(list, scan, disable, truncate ... anything) we only get errors like :
13/07/22 14:06:29 INFO httpclient.HttpMethodDirector: I/O exception
(java.net.ConnectException) caught when processing request: Connection
refused
13/07/22 14:06:29 INFO httpclient.HttpMethodDirector: Retrying request
We want to truncate some tables and gz compress some column families but
we can't do any operation.
Executing the command "jps" on the master returns :
6966 Jps
1968 JobTracker
2177 HQuorumPeer
1814 NameNode
4623 HMaster
but on the region server it only return :
5986 Jps
Rebooting didn't help ...
Any one can help ? We are not very experienced with HBase, especially
administration.
Thank you.
