Are you re-deploying over an existing installation? Is it your intention to preserve data between deployments or are you running in a testing environment? Are you clearing ZK as part of deploying a fresh cluster or are you re-using existing znodes? How did you configure replication in the shell? Can you provide those commands? I'd request debug logs from o.a.h.h.regionserver.Replication but i don't see much logging in there anyway.
Basically, can you repro this in a fresh deployment? As Himanshu points out, I'm suspect of stale configuration hanging around. On Thu, Oct 31, 2013 at 8:02 PM, Demai Ni <[email protected]> wrote: > Nick, > > thanks for looking into this problem. I attached the hbase-site.xml in > this email. Just like to point out that I have to tear down the cluster I > posted the original log. so the hbase-site.xml is from another > cluster(single-node) with the same problem. > > BTW, I did some investigation this afternoon and don't think this is a > problem within hbase code. (background: I am working within a software > team, and quite a few engineers change hbase, hadoop, and other codes > everyday)I tried out several different installations, and found out a week > ago's build with today's hbase build work just fine; but today's build with > last week's hbase doesn't. Our build includes hadoop 2, which can introduce > something problematic. > > wondering how hbase generate UUID? maybe that is something I should look > into? thanks > > Demai > > > > > > On Thu, Oct 31, 2013 at 6:20 PM, Nick Dimiduk <[email protected]> wrote: > >> Can you post your replication settings from hbase-site.xml? >> >> On Thursday, October 31, 2013, Demai Ni wrote: >> >> > hi, folks, >> > >> > I got a strange thing happening on my cluster(hbase 0.94.9) recently. I >> am >> > setting up a new cluster for replication, and didn't see the data being >> > replicated over the peer. Then, I found the following in the log of the >> > regionserver of the Master: >> > >> > 2013-10-31 13:33:03,293 INFO org.apache.hadoop.hbase.metrics: new >> MBeanInfo >> > 2013-10-31 13:33:03,300 INFO >> > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: >> Getting >> > 1 rs from peer cluster # 3 >> > 2013-10-31 13:33:03,300 INFO >> > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: >> > Choosing peer hdtest018.svl.ibm.com,60020,1383251582072 >> > 2013-10-31 13:33:03,302 INFO >> > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: >> > Replicating *b520de1d-3a18-4aec-bd45-de000e81417d* -> * >> > b520de1d-3a18-4aec-bd45-de000e81417d* >> > >> > the log is from ReplicationSource: >> > *LOG.info("Replicating "+clusterId + " -> " + peerClusterId);* >> > >> > It seems the problematic cluster is replicating to itself. >> > Any suggestion about how to look into this problem? Many thanks >> > >> > BTW, I can replicate from another cluster to this problematic one. >> > >> > Demai >> > >> > >
