sijie commented on a change in pull request #1117: BP-29: Metadata API module URL: https://github.com/apache/bookkeeper/pull/1117#discussion_r166158814
########## File path: site/bps/BP-29-metadata-store-api-module.md ########## @@ -0,0 +1,87 @@ +--- +title: "BP-29: Metadata API module" +issue: https://github.com/apache/bookkeeper/<issue-number> +state: 'Under Discussion' +release: "x.y.z" +--- + +### Motivation + +We have already abstracted all the metadata operations into interfaces. And all the bookkeeper implementations only reply on metadata interfaces, +rather than depending on zookeeper. This proposal is to organize the metadata interfaces and its implementations in a separate module and make +bookkeeper implementation only depends on metadata interfaces, not depends on zookeeper. This would a few benefits: + +- It allows supporting different metadata storages, without bringing in dependencies of metadata store implementation directly into + bookkeeper-server module. The development of different metadata storage can be done without interleaving with each other. +- It would define a clean module dependency between bookkeeper implementation and metadata api, and how bookkeeper load a different metadata + implementation. + +### Public Interfaces + +A more generic setting `metadataServiceUri` is introduced for replacing implementation specific settings `zkServers` and `zkLedgersRootPath`. + +A metadata service uri defines the location of a metadata storage. In zookeeper based implementation, the metadata service url will be +`zk://<zkServers>/<zkLedgersRootPath>`. + +This new setting in bookie configuration will be like as below: + +``` +metadataServiceUri=zk://127.0.0.1/ledgers +``` + +If we eventually support Etcd as one of the metadata storages. Then the setting in bookie configuration to use Etcd will be: + +``` +metadataServiceUri=etcd://<etcd_servers>/<etcd_key_prefix> +``` + +### Proposed Changes + +#### Configuration + +This BP proposes introducing a more generic metadata setting `metadataServiceUri` to replace implementation specific settings +`zkServers` and `zkLedgersRootPath`. All implementation specific settings should be considered moving to implementation itself. + +The `metadataServiceUri` can also be used for replacing the need of configuring `ledgerManagerFactoryClass`, `registrationClientClass` and +`registrationManagerClass`. It is unnecessarily complicated to configure multiple settings to load a specific metadata implementation. +We can just use the `scheme` field in `metadataServiceUri` to resolve which metadata implementation to use. Using uri to resolve +different driver or implementation is commonly seen at java world, for example, jdbc to support different database drivers. Also, distributedlog +uses this pattern to load different metadata driver. + +So in zookeeper based metadata implementation, the metadata service uri can be: + +- `zk+flat://127.0.0.1/ledgers`: the scheme is "zk+flat". it means a zookeeper base metadata implementation and it uses flat ledger manager. +- `zk+hierarchical://127.0.0.1/ledgers`: the scheme is "zk+flat". it means a zookeeper base metadata implementation and it Review comment: fixed ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
