2017-09-13 10:31 GMT+02:00 Jia Zhai <zhaiji...@gmail.com>: > Thanks a lot for your time Yiming and Enrico. :) > > Regarding the security, we could do it in a separate BP, and make this BP > more focus on filling up the useful endpoints. How about it? >
OK, but please consider that the more utility you add to the endpoint the more users will want to enable it and so security will be a blocker issue > > On Wed, Sep 13, 2017 at 3:30 PM, Enrico Olivelli <eolive...@gmail.com> > wrote: > > > Hi Jia, > > I am OK with the idea of having standard HTTP API, this will help > > development of (non-Java) tools. > > > > It is not clear to me if we are going to add an http API useful for > > "managing bookies" or a new HTTP REST-like Client API to BookKeeper. > > I am referring to the fact that in the proposal there are API calls to > > create ledgers and to read data. > > I think we should separate this two aspects and maybe it is better to > > address the 'bookie management' first, which is the work that Yiming > > started. > > > [jia] It is targeting for the admin portal. the existence of the ledger api > is just to simplify debugging or operations. we can eliminate the ‘create’ > endpoint. > OK so are you already thinking about specific tools to manage bookies without the bookie shell > > > > > > Another point very important to me is that if we are going to introduce > > management operations via http we have to take into account security, at > > least TLS and some kind of authentication. > > About TLS it is very simple to achieve this, every HTTP server supports > TLS > > (https) > > About authentication the problem is not so simple, Kerberos on HTTPs is > > very complex, but we need to introduce some auth mechanism. > > IMHO We can require the 'http server implementation' to implement > security > > but out of the box we have to supply basic support at least for one > > provider bundled in the distribution package. > > > [jia] The BP focuses on filling up the useful endpoints. The security will > be a separate BP. > > > > > > Some API can return very large result sets like 'list ledgers', actually > > the HTTP Server subsystem exchanges strings in memory, we will need to > > introduce some more smart way because it would be easy to bring down the > > bookie just by calling that API multiple times concurrencly (it is just > an > > example) > > > [jia] We could add pagination into all the `list` api. > Yes but it will be really tricky to implement, but please do it > > > > > IMHO The 'create ledger' API is not useful, due to the design of BK you > > have to create a ledger and then write immediately to it, I think that > such > > an API should allow the client to stream the contents of the ledger in > the > > HTTP body at least. But I think that a more stateful http API needs to be > > designed to implement a pure Http client > > > [jia] Thanks, we are not planning to implement an http client. we will > remove ‘create’ here. > OK, thanks > > > > > > Maybe we should add a more narrowed motivation and add the only useful > APIs > > to address those issues. > > For instance in my company we would like to start creating a BookKeeper > Web > > UI, so we need some way to talk to bookies and exchange data, but in this > > case I am interested in asking to bookies their status and the effective > > contents of the bookie > > > [jia] The BP proposes adding a standard naming convention for adding admin > endpoints. Feel free to propose the endpoints you would like to appear in > the http admin portal. > The ones you wrote are enough complete for me, I would like to have read-only operation in order to have a global view on the status of the system. -- Enrico > > > > > > -- Enrico > > > > > > 2017-09-13 6:09 GMT+02:00 Yiming Zang <yz...@twitter.com.invalid>: > > > > > Sure, I think the current HTTP endpoints in Twitter are only designed > for > > > Twitter specific, such as check quorum loss, check rack/region > diversity. > > > So the endpoints convention in Twitter are not the same as in the > > proposal. > > > I think it would be great to have an agreement on the API naming > design, > > so > > > I like the API design in the proposal, I think the proposal looks good > to > > > me. > > > > > > Besides, we're currently only using GET functionalities in Twitter, > but I > > > notice there're a lot of POST and PUT APIs in the proposal which could > > > change the bookie state or trigger some heavy workload. These APIs > looks > > a > > > bit risky to me if we don't have any authentication enabled (in > Twitter). > > > > > > On Tue, Sep 12, 2017 at 6:11 PM, Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > + Yiming > > > > > > > > Yiming, if you have time, please take a look at this BP. let's see if > > > > there are any conflicts with those you added for autorecovery. > > > > > > > > - Sijie > > > > > > > > On Tue, Sep 12, 2017 at 8:00 AM, Jia Zhai <zhaiji...@gmail.com> > wrote: > > > > > > > >> Hi all, > > > >> > > > >> Based on Github #278 <https://github.com/apache/bookkeeper/pull/278 > >, > > > I > > > >> have just posted a proposal regarding define BookKeeper public http > > > >> endpoints: > > > >> > > > >> https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP- > > > >> 17%3A+Define+BookKeeper+public+http+endpoints > > > >> > > > >> > > > >> Github #278 <https://github.com/apache/bookkeeper/pull/278> > > introduces > > > >> BookKeeper Http Endpoint module. However there are only two > endpoints, > > > >> which is “/heartbeat” and “/api/config/serverconfig”, defined in > #278. > > > In > > > >> order to fully leverage the http modules, The goal of this BP is to > > add > > > >> more endpoints to this modules. > > > >> > > > >> Any comments are welcome and appreciated. > > > >> > > > >> > > > >> Thanks. > > > >> > > > >> -Jia > > > >> > > > > > > > > > > > > > >
