[
https://issues.apache.org/jira/browse/BOOKKEEPER-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260337#comment-13260337
]
Sijie Guo commented on BOOKKEEPER-181:
--------------------------------------
@Flavio
> for example, by keeping a table of deleted ledgers.
I had considered the idea to keeping a table of deleted ledgers, when I
implemented metastore-based ledger manager before. It seems easy, but actually
it doesn't. The critical problem is how to remove the items from deleted
ledgers table. if we don't remove them, the table will grow and encounter the
same issue as SCAN.
removing the item should need the co-ordination between bookie servers who own
the deleted ledgers. one possible solution is using reference counting
mechanism, each deleted ledger should have a list of bookie who owed it. each
bookie delete its reference from the list when it finished garbage collecting
that ledger. the last one who deleted reference, delete the ledger item from
table.
even we have mechanism to remove items from delete table, but we still need
SCAN to get items from deleted ledgers table.
@Roger,
I am not sure that I understood the idea of dequeue. could you explain more
about it?
> Scale hedwig
> ------------
>
> Key: BOOKKEEPER-181
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-181
> Project: Bookkeeper
> Issue Type: Improvement
> Components: bookkeeper-server, hedwig-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: hedwigscale.pdf, hedwigscale.pdf
>
>
> Current implementation of Hedwig and BookKeeper is designed to scale to
> hundreds of thousands of topics, but now we are looking at scaling them to
> tens to hundreds of millions of topics, using a scalable key/value store such
> as HBase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
