Thanks Every one.It turned out that I there were a few empty wal directories corresponding to the dead region servers.Moved them out of /hbase and failed over the master.Things started working fine after that.
-Nishanth On Mon, Nov 3, 2014 at 10:25 PM, yeweichen2...@gmail.com < yeweichen2...@gmail.com> wrote: > Nishanth, > What version of HBase you are using? > > You can try clear the ZNode about regionserver list in zookeeper > /hbase/ and then restart HMaster. > > ------------------------------ > yeweichen2...@gmail.com > > > *From:* Nishanth S <nishanth.2...@gmail.com> > *Date:* 2014-11-04 02:32 > *To:* user <user@hbase.apache.org> > *Subject:* Re: Hbase Dead region Server > Thanks Pere. I just did that and still has the dead region server showing > up in Master UI as well as in status command.I have replication turned on > in hbase and seeing few issues.Below is the stack trace I am seeing. > > 2014-11-03 18:31:00,215 WARN > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't > replicate because of a local or network error: > java.io.IOException: No replication sinks are available > at > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:117) > at > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:652) > at > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:350) > 2014-11-03 18:31:00,459 WARN > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't > replicate because of a local or network error: > java.io.IOException: No replication sinks are available > at > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:117) > at > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:652) > at > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:350) > > On Mon, Nov 3, 2014 at 11:18 AM, Pere Kyle <p...@whisper.sh> wrote: > > > Nishanth, > > > > In my experience the only way I have been able to clear the dead region > > servers is to restart the master daemon. > > > > -Pere > > > > On Mon, Nov 3, 2014 at 9:49 AM, Nishanth S <nishanth.2...@gmail.com> > > wrote: > > > > > Hey folks, > > > > > > How do I remove a dead region server?.I manually failed over the hbase > > > master but this is still appearing in master UI and also on the status > > > command that I run. > > > > > > Thanks, > > > Nishan > > > > > > >
