[
https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101827#comment-14101827
]
stack commented on HBASE-11165:
-------------------------------
On the nice attached doc:
{code}
Thoughts:
Advantage of zk less assignment
1) Better API's (ls on znode vs scan table)
2) Scalability (If META is split)
a) Read/Write throughput
b) Storage (1M child znodes vs 1M
rows in META)
{code}
Can I have some pointers on how to read the above. Zk-less AM is better
because you scan a table -- you don't have to ls znodes? What is the 1M znodes
vs 1M rows about in above?
[~toffer] Is the above the basis for your "...As our experiments shows
splitting is a must for scaling."? If split meta, then more read/write
throughput? Because the meta table could be served by many machines so field
more reads/writes? The reads/writes are needed at starttime or during cluster
lifetime in your judgement? Thanks.
[~virag] Is the shrink in time from 12 mins to 5mins on zk-less assign or
forceSync=no? (or for both)?
[~enis]
bq. Do we need to bring root back? Can't we simply host all of root in
zookeeper? We expect the number of meta regions in the tens / hundreds case
right?
Root access would be different to the access of any other table if we went this
route. Everyone -- all clients, etc. -- would need to know how to do this new
access type. How would you do it in zk anyways? Would be a single znode with
all of root in it? Root would have zk enforced limits. Might be ok for a small
table that changes infrequently. Not sure about znode with 1k rows in it
(history, replicas? etc.). We'd have to test.
[~lhofhansl]
bq. We need a bootstrapping mechanism to find root anyway...
Yeah. There is zk now. Elsewhere, a quorum of masters has been proposed; you'd
go to the master to figure where everything is. That'd be a big change. 2.0.x.
> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>
> Key: HBASE-11165
> URL: https://issues.apache.org/jira/browse/HBASE-11165
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
> Attachments: HBASE-11165.zip, Region Scalability test.pdf,
> zk_less_assignment_comparison_2.pdf
>
>
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569"
> and comments on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M
> regions maybe even 50M later. This issue is about discussing how we will do
> that (or if not 50M on a cluster, how otherwise we can attain same end).
> More detail to follow.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
