[
https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115524#comment-14115524
]
stack commented on HBASE-11165:
-------------------------------
bq. ..by the way, could somebody estimate the distribution or read vs writes to
meta table, in terms of requests per second/networking traffic/disk access?
[~toffer] or [~virag] Do you have any numbers from your experiments?
bq. I believe there are 2 dimensions here, right?
There is a more immediate 3rd dimension/option and that is what we have now
where we have a single master and if it fails, backup assumes its role.
For your dimension 1., (also known as "HBASE-10296 Replace ZK with a consensus
lib(paxos,zab or raft running within master processes to provide better master
failover performance and state consistency"), it would be a nice-to-have but if
we can, lets try and avoid our having to have a HA master. As long as the
master comes back inside some reasonable window and the cluster can keep on
chugging while its figuring out its recovery, this would be a simpler deploy
than one that needs a HA master mini-cluster. Multi-master would help scale
reads but as you note, doesn't help if one massive meta region and we want to
cache it.
We'd go to partitioned masters, as I see it, if we can't make one master do.
bq. The viable combined solution ...
I suggest we look at single master with split meta served by regionservers
first?
> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>
> Key: HBASE-11165
> URL: https://issues.apache.org/jira/browse/HBASE-11165
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
> Attachments: HBASE-11165.zip, Region Scalability test.pdf,
> zk_less_assignment_comparison_2.pdf
>
>
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569"
> and comments on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M
> regions maybe even 50M later. This issue is about discussing how we will do
> that (or if not 50M on a cluster, how otherwise we can attain same end).
> More detail to follow.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
