[
https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004107#comment-14004107
]
Andrew Purtell edited comment on HBASE-11165 at 5/20/14 11:23 PM:
------------------------------------------------------------------
Looks like how the master uses ZooKeeper to manage assignments will need
updating. I created a table with 1M regions using HBASE-11211. After creating
all 1M regions, the master attempted IO of an enormous znode and aborted:
{noformat}
2014-05-20 16:07:40,402 WARN [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session 0x1461be2f6220000 for server
localhost/127.0.0.1:2181, unexpected error, closing socket connection and
attempting reconnect
java.io.IOException: Packet len6378500 is out of range!
at
org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2014-05-20 16:07:40,506 WARN [master:localhost:8100]
zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
quorum=localhost:2181,
exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase/region-in-transition
...
2014-05-20 16:07:56,754 INFO [master:localhost:8100] master.HMaster: Aborting
{noformat}
was (Author: apurtell):
Looks like how the master uses ZooKeeper to manage assignments will need
updating. I created a table with 1M regions using HBASE-11211. After creating
all 1M regions, the master attempted a write of an enormous znode and aborted
when ZK rejected it:
{noformat}
2014-05-20 16:07:40,402 WARN [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session 0x1461be2f6220000 for server
localhost/127.0.0.1:2181, unexpected error, closing socket connection and
attempting reconnect
java.io.IOException: Packet len6378500 is out of range!
at
org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2014-05-20 16:07:40,506 WARN [master:localhost:8100]
zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
quorum=localhost:2181,
exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase/region-in-transition
...
2014-05-20 16:07:56,754 INFO [master:localhost:8100] master.HMaster: Aborting
{noformat}
> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>
> Key: HBASE-11165
> URL: https://issues.apache.org/jira/browse/HBASE-11165
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
>
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569"
> and comments on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M
> regions maybe even 50M later. This issue is about discussing how we will do
> that (or if not 50M on a cluster, how otherwise we can attain same end).
> More detail to follow.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
