[
https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167927#comment-14167927
]
Elliott Clark edited comment on HBASE-12233 at 10/11/14 1:45 AM:
-
So right now I don't see the need for split meta. We found a 5x win on meta
versions. This alone means that we can handle lots more regions per cluster
with out breaking a sweat. And there's lots more that can be done without
hurting the default case.
* We haven't tried meta region replicas.
** This is where we will see the biggest wins. Right now meta gets 2x more
reads than writes on real production clusters.
** We already have the region replica stuff being worked on
** All reads to meta can already be stale so timeline consistent is good enough
for 99% of reads that are currently going to meta.
* We haven't tried allowing a shared cache infront of meta.
** It would be easy to put memcache (or redis if you already have it) in the
MetaTableAccessor and that would allow a very fast shared cache that gets
warmed by all clients.
* We haven't tried a more compact meta table representation
* We haven't tried picking smaller split keys.
** Right now we just pick a mid point. It would be possible to do something
like our index key generation to drastically reduce the size of region names.
* We haven't tried stripe compactions on meta.
* We haven't gotten block encoding working for meta.
Each of those things are largish wins that haven't been tried, and each of
those things helps every user. While split meta helps maybe 0.1% of HBase's
users.
Until the less invasive more positive options are tried and shown to not be
enough, it just feels like everyone is trying to split meta without a concrete
need.
Splitting meta doesn't make us able to scale to the sky. The NN is still a
bottle neck on cluster size. NameNodes fall over way before we will need a
split meta. Even if someone gets NN to scale to infinity, the network becomes
a bottleneck. HDFS clusters can only get so large before they need to be
federated (and HBase doesn't tolerate federation). So making every user and
every cluster feel the pain of increased complexity and mttr so that we the
HBase developers can perform a thought experiment is just not something I see
the benefit in right now.
I am for sure against bringing root back in for branch-1 ( -1 ). It's just way
too late in the stabilization period to not set us way back.
was (Author: eclark):
So right now I don't see the need for split meta. We found a 5x win on meta
versions. This alone means that we can handle lots more regions per cluster
with out breaking a sweat. And there's lots more that can be done without
hurting the default case.
* We haven't tried meta region replicas.
** This is where we will see the biggest wins. Right now meta gets 2000x more
reads than writes on real production clusters.
** We already have the region replica stuff being worked on
** All reads to meta can already be stale so timeline consistent is good enough
for 99% of reads that are currently going to meta.
* We haven't tried allowing a shared cache infront of meta.
** It would be easy to put memcache (or redis if you already have it) in the
MetaTableAccessor and that would allow a very fast shared cache that gets
warmed by all clients.
* We haven't tried a more compact meta table representation
* We haven't tried picking smaller split keys.
** Right now we just pick a mid point. It would be possible to do something
like our index key generation to drastically reduce the size of region names.
* We haven't tried stripe compactions on meta.
* We haven't gotten block encoding working for meta.
Each of those things are largish wins that haven't been tried, and each of
those things helps every user. While split meta helps maybe 0.1% of HBase's
users.
Until the less invasive more positive options are tried and shown to not be
enough, it just feels like everyone is trying to split meta without a concrete
need.
Splitting meta doesn't make us able to scale to the sky. The NN is still a
bottle neck on cluster size. NameNodes fall over way before we will need a
split meta. Even if someone gets NN to scale to infinity, the network becomes
a bottleneck. HDFS clusters can only get so large before they need to be
federated (and HBase doesn't tolerate federation). So making every user and
every cluster feel the pain of increased complexity and mttr so that we the
HBase developers can perform a thought experiment is just not something I see
the benefit in right now.
I am for sure against bringing root back in for branch-1 ( -1 ). It's just way
too late in the stabilization period to not set us way back.
Bring back root table
-
Key: HBASE-12233
URL: https://issues.apache.org/jira/browse/HBASE-12233
