[
https://issues.apache.org/jira/browse/BOOKKEEPER-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261414#comment-13261414
]
Ivan Kelly commented on BOOKKEEPER-181:
---------------------------------------
{quote}
I don't think I was being clear enough. Actually this is not what I was saying.
My point was that ledger garbage collection can be implemented without scan,
simplifying the kv API. Nor would I want to put the dequeue into the API (as
you point out this makes no sense, and I agree). The dequeue is an
application-specific implementation technique for implementing ledger deletion
which doesn't rely on SCAN but only on get/set/delete/CAS.
{quote}
My underlying point was, that if the metastore interface cannot be used,
unmodified for bookkeeper and for hedwig, then motivation for metastore goes
away. Sijie came up with metastore to have a means to scale hedwig+bk by
implementing a single interface. What you propose makes this no longer the
case. If the plugin developer has to implement the metastore and the
metastore+scan interfaces, they may as well be implementing the
LedgerManager(bk) and the TopicManager(hw) interfaces.
Whats more, if scan isn't available, it will be possible to use a deque for GC
on BK, but this would be more comfortably done at the level of LedgerManager.
If it is done at the level of metastore, then the MetaStoreLedgerManager will
need to implement using deque. If any implementation of metastore does support
scan(such as HBase or an SQL backend), it can no longer be used, as
MetaStoreLedgerManager uses the deque.
> Scale hedwig
> ------------
>
> Key: BOOKKEEPER-181
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-181
> Project: Bookkeeper
> Issue Type: Improvement
> Components: bookkeeper-server, hedwig-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: hedwigscale.pdf, hedwigscale.pdf
>
>
> Current implementation of Hedwig and BookKeeper is designed to scale to
> hundreds of thousands of topics, but now we are looking at scaling them to
> tens to hundreds of millions of topics, using a scalable key/value store such
> as HBase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
