On Thu, Mar 15, 2018 at 5:39 PM, 张铎(Duo Zhang) <palomino...@gmail.com> wrote:
> > But for other stuffs for replication, such as the WAL files we need to > replicate, the row key will be a peer id, and multiplies the server name, > and maybe also the file name? This is a completely different thing. If we > put this in meta, the row key will be messed up... > > Is there some artifice that would allow us shape this info so it fit the meta schema: e.g. a table that we do not assign (special namespace? special tablename? hbase:replication?). Then perhaps the regions in meta have peer as the start row.... etc. St.Ack > 2018-03-16 3:01 GMT+08:00 Stack <st...@duboce.net>: > > > On Wed, Mar 14, 2018 at 11:55 PM, OpenInx <open...@gmail.com> wrote: > > > > > Hi : > > > > > > (Paste from https://issues.apache.org/jira/browse/HBASE-20166? > > > focusedCommentId=16399886&page=com.atlassian.jira. > > > plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16399886) > > > > > > There's a really big problem here if we use table based replication to > > > start a hbase cluster: > > > > > > For HMaster process, it works as following: > > > 1. Start active master initialization . > > > 2. Master wait rs report in . > > > 3. Master assign meta region to one of the region servers . > > > 4. Master create hbase:replication table if not exist. > > > > > > > > We have to have a new system table? Can't we add a column family on > > hbase:meta that keeps offsets? We've done the work to make sure > hbase:meta > > is up before everything else. It has its own WALs so we can split these > > ahead of user-space WALs, and so on. We've not done the work to for > > hbase:replication or hbase:namespace, hbase:acl... etc. > > > > Means more loading on hbase:meta and it is going to get bigger but I'd > > rather work on splitting meta than on figuring how to preassign > > miscellaneous system tables; one-per-feature. > > > > > > > > > But the RS need to finish initialize the replication source & sink > before > > > finish startup( and the initialization of replication source & sink > must > > > finish before opening region, because we need to listen the wal event, > > > otherwise our replication may lost data), and when initialize the > source > > & > > > sink , we need to read hbase:replication table which hasn't been > avaiable > > > because our master is waiting rs to be OK, and the rs is waiting > > > hbase:replication to be OK ... a dead loop happen again ... > > > > > > After discussed with Guanghao Zhang offline, I'm considering that try > to > > > assign all system table to a rs which only accept regions of system > table > > > assignment (The rs will skip to initialize the replication source or > sink > > > )... > > > > > > > > Can we avoid this sort of special-casing? > > > > St.Ack > > > > > > > > > I've tried to start a mini cluster by setting > > > hbase.balancer.tablesOnMaster.systemTablesOnly=true > > > & hbase.balancer.tablesOnMaster=true , it seems not work. because > > > currently > > > we initialize the master logic firstly, then region logic for the > HMaster > > > process, and it should be ... > > > > > > > > > Any suggestion ? > > > > > >
