[
https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002864#comment-14002864
]
Francis Liu commented on HBASE-11165:
-------------------------------------
{quote}
autotuning memstore sizes, and lazy allocation (as Andy says), or sharing
memstores
{quote}
If we increase the memstore multiplier to a high number won't that be a rough
simulation of this? Also if the writes are uniformly distributed across regions
then sharing is not needed.
{quote}
make large regions more workable, splits, compations, etc
{quote}
It's seems to me it's in HBase's DNA to have small regions. My gut tells me
that it would take less effort to support more regions.
{quote}
allow more RAM to be used by region server (off heap memstores)
{quote}
Or support larger heap :-)
{quote}
allow smaller units of computation in M/R
{quote}
We generally need a smarter way of calculating splits. ie Control number of
tasks accessing an RS. If only there was some integration between an NM and RS.
{quote}
split META? And then colocate with multiple HMasters?
{quote}
IMHO HBase should be horizontally scalable with regards to # of regions. If I
have too many regions I should be able to add more machines (ie
master/regionserver). Currently at ~68k regions, it's consuming about ~200MB.
Extrapolating at 6M it's 20GB and 60M it's 200GB.
> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>
> Key: HBASE-11165
> URL: https://issues.apache.org/jira/browse/HBASE-11165
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
>
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569"
> and comments on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M
> regions maybe even 50M later. This issue is about discussing how we will do
> that (or if not 50M on a cluster, how otherwise we can attain same end).
> More detail to follow.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
