[
https://issues.apache.org/jira/browse/BOOKKEEPER-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151913#comment-13151913
]
Ivan Kelly commented on BOOKKEEPER-39:
--------------------------------------
I think the main objection was
{quote}
we use zk sequential node to get unique incremental ledger id. if we get ledger
id then split it, we also have a zk node (which is for generating ledger ids)
which has too many children.
{quote}
This isn't the case, as the sequential node doesn't have to exist for very
long. It can be deleted immediately after. So the too many children isn't an
issue. I'll try to implement this on your interface today.
{code}
String newId = zk.create("/ledgers/idgen/ID-", SEQUENTIAL | EPHEMERAL);
long id = Long.valueOf(newId.replace("ID-"));
zk.delete(newId, -1);
{code}
> Bookie server failed to restart because of too many ledgers (more than
> ~50,000 ledgers)
> ---------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-39
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-39
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.0.0
>
> Attachments: bookkeeper-39.patch, bookkeeper-39.patch_v2
>
>
> If we have ~500,000 topics in hedwig, we might have more than ~500,000
> ledgers in bookkeeper (a topic has more than 1 ledger). So when the bookie
> server restarted, a logfile GC thread is started, which will call
> zk.getChildren to fetch all ledgers, and it failed because of package length
> limitation.
> 2011-08-01 01:18:46,373 - ERROR
> [main-EventThread:EntryLogger$GarbageCollectorThread$1@164] - Error polling
> ZK for the available ledger nodes:
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
> = ConnectionLoss for /ledgers
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1519)
> at
> org.apache.bookkeeper.bookie.EntryLogger$GarbageCollectorThread$1.processResult(EntryLogger.java:162)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:592)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:481)
> 2011-08-01 01:18:46,373 - WARN [main-EventThread:Bookie$1@242] - ZK client
> has been disconnected to the ZK server!
> 2011-08-01 01:18:47,278 - WARN
> [main-SendThread(perf13.platform.mobile.sp2.yahoo.com:2181):ClientCnxn$SendThread@980]
> - Session 0x131833dec850034 for server
> perf13.platform.mobile.sp2.yahoo.com/98.139.43.86:2181, unexpected error,
> closing socket connection and attempting reconnect
> java.io.IOException: Packet len9976413 is out of range!
> at
> org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:78)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:264)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:958)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
