jvrao commented on a change in pull request #1117: BP-29: Metadata API module
File path: site/bps/BP-29-metadata-store-api-module.md
@@ -0,0 +1,87 @@
+title: "BP-29: Metadata API module"
+state: 'Under Discussion'
+We have already abstracted all the metadata operations into interfaces. And
all the bookkeeper implementations only reply on metadata interfaces,
+rather than depending on zookeeper. This proposal is to organize the metadata
interfaces and its implementations in a separate module and make
+bookkeeper implementation only depends on metadata interfaces, not depends on
zookeeper. This would a few benefits:
+- It allows supporting different metadata storages, without bringing in
dependencies of metadata store implementation directly into
+ bookkeeper-server module. The development of different metadata storage can
be done without interleaving with each other.
+- It would define a clean module dependency between bookkeeper implementation
and metadata api, and how bookkeeper load a different metadata
+### Public Interfaces
+A more generic setting `metadataServiceUri` is introduced for replacing
implementation specific settings `zkServers` and `zkLedgersRootPath`.
+A metadata service uri defines the location of a metadata storage. In
zookeeper based implementation, the metadata service url will be
+This new setting in bookie configuration will be like as below:
+If we eventually support Etcd as one of the metadata storages. Then the
setting in bookie configuration to use Etcd will be:
+### Proposed Changes
+This BP proposes introducing a more generic metadata setting
`metadataServiceUri` to replace implementation specific settings
+`zkServers` and `zkLedgersRootPath`. All implementation specific settings
should be considered moving to implementation itself.
+The `metadataServiceUri` can also be used for replacing the need of
configuring `ledgerManagerFactoryClass`, `registrationClientClass` and
+`registrationManagerClass`. It is unnecessarily complicated to configure
multiple settings to load a specific metadata implementation.
+We can just use the `scheme` field in `metadataServiceUri` to resolve which
metadata implementation to use. Using uri to resolve
+different driver or implementation is commonly seen at java world, for
example, jdbc to support different database drivers. Also, distributedlog
+uses this pattern to load different metadata driver.
+So in zookeeper based metadata implementation, the metadata service uri can be:
+- `zk+flat://127.0.0.1/ledgers`: the scheme is "zk+flat". it means a zookeeper
base metadata implementation and it uses flat ledger manager.
+- `zk+hierarchical://127.0.0.1/ledgers`: the scheme is "zk+flat". it means a
zookeeper base metadata implementation and it
+ uses hierarchical ledger manager.
+- `zk+longhierarchical://127.0.0.1/ledgers`: the scheme is "zk+flat". it means
a zookeeper base metadata implementation and it
Nit: zk+longhierarchical not zk+flat.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
Apache Git Services