I am configuring a hbase cluster for the first time. The issue I am seeing is, when i setup a single server acting as master and region server, everything seems to be working fine. If i start region servers on 2 other boxes, in the base ui i see duplicate region servers. I have read that this can be normal, and base will eventually "sort things out". But this is showing the ip (with unknown version) and hostname (with correct base version), and showing an error of 2 nodes with inconsistent version.
I also see an error on the UI that one of the regions is in transition and state=PENDING_OPEN. I looked at the logs files and see the following error: 2016-07-18 20:38:10,329 WARN [RS_OPEN_META-hb02:16020-0] zookeeper.ZKAssign: regionserver:16020-0x255e5cd7f190042, quorum= zk01.domain.com:2181,zk02.corp.lumsb.com:2181,zk03.domain.com:2181, baseZNode=/hbase Attempt to transition the unassigned node for 1588230740 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING failed, the server that tried to transition was 10.110.2.128,16020,1468874283476 not the expected hb02.domain.com,16020,1468874283476 This makes me think the duplicate error and the region transition errors may be related. I'm running the same hbase-site.xml on every node. I'm very confused why this just works with single server, but when i do extra servers, it fails. Any ideas? Thank you in advance.
