Boris, what does hbck say?
We have had this issue a couple times before. To fix it I had to stop the
cluster, run offline meta repair tool,
delete zk-store on each zk quorum node
Offline Meta repair tool will not work if there are inconsistencies in HBase
- you better try hbase hbck
-fixAll first.
Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ,www.carrieriq.com <http://www.carrieriq.com/>
e-mail:vrodionov@...
<http://gmane.org/get-address.php?address=vrodionov%2dSvj7bELwklqcm8Fc2pXOzQ%40public.gmane.org>
Hbck says "0 inconsistencies detected".
I stopped hbase cluster, deleted zk-database on all quorum nodes, ran "hbase
org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair",
and got "INFO util.HBaseFsck: Success! .META. table rebuilt.".
After that, cluster continued crashing during auto-loadbalancing.
--
Best regards,
Boris Emelyanov.