[
https://issues.apache.org/jira/browse/HBASE-22938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17175939#comment-17175939
]
Duo Zhang commented on HBASE-22938:
-----------------------------------
As splittable meta is not a technical problem now, let's revive this issue.
If we want to purge the zk dependency, the replication peers and wal files
should be moved to other storage systems. In HBASE-15867, we proposed to store
them in a system table. But then, there is dead lock issue, as when creating a
new wal writer, we need to register it into the replication tracking system,
but the registration itself needs to write a WAL out as we use a system table
now right? Think of the start up of a fresh new cluster with replication
enabled...
A possible solution is to also make special wal writer for system tables, as we
do not need to replicate system tables, so we can skip registering the wal
writer. Then there is another problem, that we have to add new steps to deal
with system table assignment in SCP, as if the replication system table is not
online, we can not create new wal writers...
So I think a possible way is to fold the replication system table to meta
table, with a separated region, and also a separated family. Since we already
have steps in SCP to deal with meta regions first, it does not add new
complexity to our recovery code.
Of course there is a downside for storing replication information in a system
table or meta table, as we need to make an assumption that these tables will
not be replicated. This will make the read replicas feature can not 100% work
for meta and system tables, as we can not use in-cluster replication for them
to spread the edits to the secondary replicas any more.
Thanks.
> Fold all the system tables to hbase:meta
> ----------------------------------------
>
> Key: HBASE-22938
> URL: https://issues.apache.org/jira/browse/HBASE-22938
> Project: HBase
> Issue Type: Brainstorming
> Reporter: Duo Zhang
> Priority: Major
>
> Quote my post on HBASE-15867 here, on how to deal with the dead lock when we
> want to store replication queues to hbase:replication table.
> {quote}
> We could add a special prefix in the row key for different system tables, and
> make a special family for it. For example, for all the records in hbase:acl,
> we could introduce a prefix like ':::acl:::', since we do not allow ':' in
> either namespace or table name, so it will not conflict with the existing
> table related records. And the family could be namd as 'acl'.
> And we could make a special split policy that only splits at these special
> prefixs, so it will not break any assumptions so far, as all the records for
> the 'system table' are in the same region.
> {quote}
> And I think there are also other advantages, for example the start up logic
> can be greatly simplified.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
