[
https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120970#comment-14120970
]
stack commented on HBASE-11165:
-------------------------------
[~eclark]
bq. ...If we can get into the same scaling range as HDFS's namenode then I
don't see the urgency to split meta.
There is a largish gap at the moment.
In-memory representation of cluster is not the immediate barrier to scaling.
Getting more r/w iops, cache, and putting off mad compactions on a single big,
critical meta region are the more immediate priorities. Row caching and read
replicas are just as crazy pants as splitting meta but in the end you are only
'solving' the read i/o issue and leaving aside how we'd deal w/ fat meta, write
iops, and cache -- nevermind we have all eggs in one basket.
bq. ...doing more active writes then meta on a stable cluster
Stable cluster is uninteresting. Its the stop cluster, install software,
restart cluster in minimal time is where all the fun is.
bq. ...changing the deployment or complexity of normal users.
Yeah, this is an issue. Most people will not want/need split meta. A 'go big'
button would be messy...
[~mantonov] Let me put up doc. for all to throw darts at. Has a few stats in
it.
> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>
> Key: HBASE-11165
> URL: https://issues.apache.org/jira/browse/HBASE-11165
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
> Attachments: HBASE-11165.zip, Region Scalability test.pdf,
> zk_less_assignment_comparison_2.pdf
>
>
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569"
> and comments on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M
> regions maybe even 50M later. This issue is about discussing how we will do
> that (or if not 50M on a cluster, how otherwise we can attain same end).
> More detail to follow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
