Thanks Sijie for discussing on implementation details. 1) how to delete a ledger? so you need to take care of telling from a leave znode from a inner znode in your implementation. 2) taking care of iterating the tree in order, since Scan And Compare gc algorithm relies on the order. But if we evolved to improved gc algorithm, this is not be a problem. maybe we need to start pushing the changes for improved gc algorithm into trunk after 4.2.0 released. -----------------------------------
You are right, we can distinguish whether inner znode is deleted or not by its data size simply. And iterate the tree in order only requires a BFS process, but it shows that previous ScanAndCompare GC may require one more read for each inner znode. 3) could ZooKeeper#getChildren carry 8192 children? Flavio, do you have any number about it? or maybe we need some experiments to ensure it works. ----------------------------------- I'll make experiments after the coding. I would suggest that you could start this work as a new ledger manager w/o affecting other ledger managers. And apply the idea back to HierarchicalLedgerManager later if possible. ----------------------------------- Ok, I'll make a new LedgerManager. Thanks, Jiannan On 1/18/13 4:26 PM, "Sijie Guo" <[email protected]> wrote: >Thanks Jiannan for providing and organizing the discussions. > >currently bookkeeper could not handle id confliction perfectly so far >especially for the case 3 Flavio mentioned. So we heavily replied on >an implementation of id generator need to generate unique ledger id, which >means the id generation requires a centralized place to track or >co-ordinate. (e.g zookeeper). It is not perfect, but better as Jiannan >proposed. But since Jiannan has provided the patch to separate id >generation from ledger manager, it would be helpful to the community to >evaluate better and fast id generation algorithm. So let's improve it by >iterations. > >> ledger metadata management. > >thinking more about radix tree Jiannan proposed. several comments on it: > >1) how to delete a ledger? so you need to take care of telling from a >leave >znode from a inner znode in your implementation. >2) taking care of iterating the tree in order, since Scan And Compare gc >algorithm relies on the order. But if we evolved to improved gc algorithm, >this is not be a problem. maybe we need to start pushing the changes for >improved gc algorithm into trunk after 4.2.0 released. >3) could ZooKeeper#getChildren carry 8192 children? Flavio, do you have >any >number about it? or maybe we need some experiments to ensure it works. >4) if this change wants to be applied on HierarchicalLedgerManager, it >would be better to consider how to make the change smoothly, since it is >different organized format. > >from the above comments, I would suggest that you could start this work as >a new ledger manager w/o affecting other ledger managers. And apply the >idea back to HierarchicalLedgerManager later if possible. > >-Sijie
