Re: [DISCUSS] Slack Channel for BookKeeper

[Dustin Castor]

Agreed! Or an IRC. 

On Monday, July 24, 2017, 6:50:22 PM PDT, Jia Zhai  wrote:

 It is great to have a slack channel. It make things more effective and
smooth.

On Tue, Jul 25, 2017 at 8:11 AM, Sijie Guo  wrote:

> Hi all,
>
> What do you guys all think about having a dedicated slack channel for
> informal discussion for the community? There are a handful of Apache
> projects are doing that already, there are also ways to have a bot that
> sends daily digest of the conversation to the mailing lists (to keep the
> records in ASF infrastructure).
>
> As the followup steps for merging DL into BookKeeper, we are transferring
> the DL slack channel to BookKeeper PMC. We can just make it a BK slack
> channel, and have different channels under it for different discussions.
>
> Thoughts?
>
> - Sijie
>

Re: Bookie Http Endpoint

[Dustin Castor]

I do agree that we need a contractual way of managing the server lifecycle via 
an interface -- identical to how we do it with stat providers. Personally I 
don't think we can really have contracts for each and every method we all want 
from an interface, so not so sure how far the interface can go in offering 
uniform methods for what we all want an endpoint to do. Sure, we all want to 
get configurations, but what if we wanted to to do something else (e.g., 
run-time config changes of certain parameters) and someone else didn't?

Regarding paths, I did like the simplicity of the Jersey implementation. E.g., 
put this right above class to denote a path:
@Path("/resources")public class Rest{
And this above a method:@GET
@Path("/v1/configurations")
@Produces(MediaType.APPLICATION_JSON)
public String getConfiguration(@QueryParam("config") String config) { } 
This could be hit via something like 
[host]:[port]/resources/v1/configurations?config=BookiePort

Agreed that ultimately so long as they work, I don't really care which library 
is used. 




On Wednesday, July 19, 2017, 2:52:38 AM PDT, Sijie Guo <guosi...@gmail.com> 
wrote:

On Wed, Jul 19, 2017 at 2:55 PM, Enrico Olivelli <eolive...@gmail.com>
wrote:

> 2017-07-19 3:27 GMT+02:00 Yiming Zang <yz...@twitter.com.invalid>:
>
> > Hi all,
> >
> > We also have a working implementation in Twitter, we use Http Endpoint
> > mostly for getting quorum loss information, underreplicated ledgers,
> manage
> > bookie status etc.
> >
> > The HTTP server in Twitter is implemented in such a way that it has a
> > generic skeleton which can easily embed with different HTTP frameworks
> > (Vertx, Undertow, TwitterServer, etc), because all these frameworks have
> > something in common, they all have Http Handler, we only need to
> implement
> > the functions for the Handler, and then bind the Handler to a specific
> > endpoint in the router. We intend to keep the code simple and neat, easy
> to
> > extend, and keep the implementation flexible. There's no limitation here.
> >
> > The pull request https://github.com/apache/bookkeeper/pull/210 is
> > basically
> > the skeleton of the HTTP server in Twitter. If this is what's needed in
> > OSS, I'm happy to keep working on the pull request and push the changes
> to
> > master.
> >
>
>
> I think that the main point is that we need to draft a "standard" HTTP API
> for the Bookie, then we can make several implementations of such API.
> Once we have a common API it will be really easy to create tools and
> integrate BK with other systems, like we are already doing with the Stats
> API.
>
> For me a great thing would be to have a ready to use HttpServlet, which is
> the "standard" in the Java Web would and can be deployed on every Java Web
> container.
> A JAX-RS resource could be good too, but it needs more support from  the
> container.
>
> In the near future we (at Diennea) wnat to start developing a global WebUI
> for BookKeeper, which will show the status of the cluster, and having HTTP
> endpoints in Bookie will ease this work
>
> Does anyone want to start a Wiki page ?
>

Enrico, can we hold on starting a wiki page?

Currently I see three different approaches on this topic. An email
discussion or a google doc might be better at this point.

I think people have different opinions on what frameworks to use for
implementing a http service. Can we do something like what we did for stats
provider. The contract that we need to make for the http service is:

1. what are the uri endpoints for different functions?
2. for each endpoint, what is request method (put/get/post), what is the
request (json) and what is the response (json)?

If all the implementations meet the requirements, it doesn't matter it is
implemented in which frameworks. This is also how web services work for
3rd-party developers.

If using this rule, I don't know see any major problems from Yiming's
approach. It is actually quite clean an approach.

- all the endpoints, requests and response spec go to the HttpServer
interface (
https://github.com/apache/bookkeeper/pull/210/files#diff-3d2be384942070a58132c5392fc105c3).
It is the contract between the bookie http server and the 3rd party
developers.
- all the implementations are just implementing the contract. if there is a
requirement to have a servlet implementation, it should be also easy to
implement. same applies for jetty + JAX-RS.


So can we agree on the contract and defer the implementation to different
organizations (like what we did for stats providers)?

^^ Yiming, Dustin, Enrico


- Sijie








>
> -- Enrico
>
>
>
>
>
> > Best,
> >
> > Yiming
> >
> > On Mon, Jul 17, 2017 at 12:52 PM, 

Re: Bookie Http Endpoint

[Dustin Castor]

Hey all,
So I did define a little endpoint that can be used for multiple things on our 
end. Basically it spins up a jetty server, attaches a handler, and then maps 
that handler to a class. Within that class, you can add methods and map them 
using the Jersey API where you'd "decorate" methods with things like @PATH, 
@PARAM, etc. Currently we use it to return configs (all, or one that you 
specify in the URL). Realistically, it can be used for whatever you want if you 
define a method to handle the endpoint.
The only limitation that I've encountered is that the Jersey API cannot 
instantiate objects without a no arg constructor, so basically if you want to 
have application context, it needs to be something static and passed in to this 
class as an object. 
I'd be happy to consolidate or lend a hand here. If this sounds like something 
that isn't too limited (as per what I described) for general use, then I'd be 
happy to work on introducing it generally. 
Thanks,
Dustin



On Monday, July 17, 2017, 12:45:51 PM PDT, Venkateswara Rao Jujjuri 
 wrote:

+ Dustin

On Mon, Jul 17, 2017 at 12:30 PM, Sijie Guo  wrote:

> + Yiming
>
> I would suggest the people who already started the implementations (either
> from Twitter or Salesforce) taking the lead. because they either already
> had the implementation or are working on the implementation. I think the
> goal is to consolidate existing implementations to see if we can have a
> unified effort on this.
>
> - Sijie
>
> On Mon, Jul 17, 2017 at 5:39 PM, Enrico Olivelli 
> wrote:
>
> > Hi all,
> > A discussion has been started about introducing an HTTP endpoint to the
> > bookie.
> > There has been a proposal from Twitter and this is the patch
> > https://github.com/apache/bookkeeper/pull/210
> > On Salesforce there is an ongoing implementation too.
> > I have added that we should provide a Servlet based approach or at least
> > define a public API.
> > We should start a discussion and maybe a BP.
> > We need a leader for the discussion
> >
> > Any volunteer?
> > Enrico
> > --
> >
> >
> > -- Enrico Olivelli
> >
>



-- 
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi