Hi Matteo, I think the main proposal that Florentin and Steven proposed is to define a `scope` for each restful api. The scope indicates where a client should send the http request. The current restful APIs can be categorized into three scopes:
- broker: send to this broker. - cluster: send to any broker in this cluster. - instance (constellation as proposed here): send to any broker at any cluster in the instance. We also encountered some of the pain points that Florentin and Steven encountered when developing Pulsar Manager. Because the lack of scope in restful API, people makes mistakes when send the http request. So we ended adding a backend in the Pulsar Manger and inject scope information by ourselves and use it to get the right service url to send the http requests. I wish we can look into resolve this problem in a newer version. The proposed PIP tries to organize the restful apis in a better way to make people easier figure out where they should send the http request. Whether to rename "instance" to "constellation" can probably be moved to a separate PIP. We can discuss it in a separate thread. - Sijie On Thu, Oct 31, 2019 at 12:21 AM Matteo Merli <matteo.me...@gmail.com> wrote: > Hi Florentin, > > I have bit of trouble understanding the motivation and scope of this > proposal. There seems to be a mix of multiple different items. > > > Also, Pulsar evoled to the ability to manage multiple clusters. > > Pulsar had the ability of managing multiple clusters from day 0 (or I > should say day -1200): multiple clusters in a single logical > "instance" spanning across multiple geo-distributed data centers. > > It has not "evolved" into that. > > > For example to administrate a namespace, we used the following route > > v2/namespaces/:tenant-id/:namespace-id. This could be confusing, > > because we intend to manipulate a namespace under a tenant scope, > > which still requires here to give the tenant identifier in addition of > the namespace identifier. > > A namespace *is* identified by TENANT_ID/NAMESPACE_ID. The REST API > just reflects that fact. > > A namespace only exist in the context of a tenant. Through ACLs, one > can decide to give permission to other tenants to > produce/consume though that doesn't change the fundamental property > that a namespace is always scoped inside a tenant. > > > > offer a more hierarchical routing approach reflecting the Pulsar semantic > > It's not clear to me what hierarchical here means or in which way the > current API is not hierarchical. > > > proposes to officially name an ensemble of Pulsar cluster : a > Constellation > > I'm not sure anyone can immediately grasp what a "Constellation" is in > this context. We've been using "instance" in the > past and there was a proposal to use "Federation". I'm open for better > proposal though I think we should strive to use a > term that is commonly used in the context of a "group of logically > related clusters". > > Also it would have to be clearldefined what an "an ensemble of Pulsar > clusters" is. > > > to simplify the user experience for : > > Topic management between persistent and non-persistent. > > This was not changed on REST API for compatibility (just on JAVA admin > and CLI), though I I think that fixing it should not require > any breaking changes, just adding a new handler and deprecating the > current ones. > > > cluster management inside a Pulsar Constellation > > What does this means? It's already possible to manage the metadata of > the clusters since the configuration metadata is stored in the global > zookeeper instance. > > > > >>> v4/metricsbroker metrics (both broker and worker) in prometheus format > > Prometheus metrics should be exposed at /metrics only > > >>> v4/broker/statusretrieve broker statusbroker > > What's the broker status? And why cannot a new handler just be added > to the current v2 API? > > > >>> v4/clustersList and create cluster instances constellation > > How's that different from admin/v2/clusters? > > > > > On Wed, Oct 30, 2019 at 8:15 AM Florentin Dubois > <florentin.dub...@corp.ovh.com> wrote: > > > > Hello, > > > > > > We (Steven and I) have written a draft of pulsar improvement proposal > about the admin api. > > > > > > We would like to get some feedback/review about it :D > > > > > > The pip is here: > https://gist.github.com/FlorentinDUBOIS/4f98c0f71bf6514309b0414290b48e6a > > > > > > Feel free to contact me on slack or on this mailing list, if you have > any questions ;). > > > > > > Have a nice day! > > > > > > --- > > > > Florentin Dubois - OVH Group > > > > IO - Tech Lead > > > > florentin.dub...@corp.ovh.com > > > > +33 6 58 37 43 83 > > >
