I also think it would be good to have a REST client that can interact with the new management API's coming down the pipe.
On Tue, Oct 4, 2016 at 10:35 PM, Edoardo Comar <eco...@uk.ibm.com> wrote: > Harsha > thanks for opening the discussion on this KIP. > > While I understand he founding members' stand that the Kafka project can > not expand its surface to a large number of clients, > I strongly agree with your well explained points below and support your > KIP. > > A REST API is not just on the same level as any client, it's a basic > building block of open web technologies. > It's the API that most of our first time user want to try out (or that > would be users ask for and expect to be there). > > A REST API for Kafka and a robust server implementation > under the open governance of the Apache community would be most welcome. > > +1 > > Edo > -------------------------------------------------- > Edoardo Comar > IBM MessageHub > > IBM United Kingdom Limited Registered in England and Wales with number > 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 > 3AU > > Harsha Chintalapani <ka...@harsha.io> wrote on 02/10/2016 21:23:15: > > > From: Harsha Chintalapani <ka...@harsha.io> > > To: dev@kafka.apache.org > > Date: 02/10/2016 21:23 > > Subject: Re: [DISCUSS] KIP-80: Kafka REST Server > > > > Neha & Jay, > > We did look at the open source alternatives. Our > concern > > is what's the patch acceptance and adding features/ bug-fixes to the > > existing project under a Github (although it's licensed under Apache > 2.0). > > It would be great if that project made available under Apache and driven > by > > the community. Adding to the above, not all Kafka users are interested > in > > using the Java client API, they would like to have simple REST API where > > they can code against using any language. I do believe this adds value > to > > Apache Kafka in itself. > > > > "For 1, I don't think there is value in giving in to the NIH syndrome > and > > reinventing the wheel. What I'm looking for is a detailed comparison of > the > > gaps and why those can't be improved in the REST proxy that already > exists > > and is actively maintained." > > > > We are not looking at this as NIH. What we are worried about is a > project > > that's not maintained in a community. So the process of accepting > patches > > and priorities is not clear, and it's not developed in Apache community. > > Not only that, existing REST API project doesn't support new client API > and > > hence there is no security support either. > > We don't know the timeline when that's made available. We would like to > add > > admin functionality into the REST API. So the Roadmap of that project is > > not driven by Apache. > > > > > > "This doesn't materially have an impact on expanding the usability of > > Kafka. In my experience, REST proxy + Java clients only cover ~50% of > all > > Kafka users, and maybe 10% of those are the ones who will use the > REST > > proxy. The remaining 50% are non-java client users (C, python, go, > node > > etc)." > > > > REST API is most often asked feature in my interactions with Kafka > users. > > In an organization, There will be independent teams who will write their > > Kafka clients using different language libraries available today, and > > there is no way to standardize this. Instead of supporting several > > different client libraries users will be interested in using a REST API > > server. The need for a REST API will only increase as more and more > users > > start using Kafka. > > > > "More surface area means more work to keep things consistent. Failure > > to do that has, in fact, hurt the user experience." > > Having myriad Kafka client GitHub projects that support different > languages > > hurts the user experience and pushes burden to maintain these libraries. > > REST API is a simple code base that uses existing java client libraries > to > > make life easier for the users. > > > > Thanks, > > Harsha > > > > On Sat, Oct 1, 2016 at 10:41 AM Neha Narkhede <n...@confluent.io> wrote: > > > > > Manikumar, > > > > > > Thanks for sharing the proposal. I think there are 2 parts to this > > > discussion - > > > > > > 1. Should we rewrite a REST proxy given that there is a > feature-complete, > > > open-source and actively maintained REST proxy in the community? > > > 2. Does adding a REST proxy to Apache Kafka make us more agile and > maintain > > > the high-quality experience that Kafka users have today? > > > > > > For 1, I don't think there is value in giving in to the NIH syndrome > and > > > reinventing the wheel. What I'm looking for is a detailed comparison > of the > > > gaps and why those can't be improved in the REST proxy that already > exists > > > and is actively maintained. For example, we depend on zkClient and > have > > > found as well as fixed several bugs by working closely with the people > who > > > maintain zkClient. This should be possible for REST proxy as well, > right? > > > > > > For 2, I'd like us to review our history of expanding the surface area > to > > > add more clients in the past. Here is a summary - > > > > > > - This doesn't materially have an impact on expanding the usability > of > > > Kafka. In my experience, REST proxy + Java clients only cover ~50% > of > > > all > > > Kafka users, and maybe 10% of those are the ones who will use the > REST > > > proxy. The remaining 50% are non-java client users (C, python, go, > node > > > etc). > > > - People are a lot more excited about promising to contribute while > > > adding the surface area but not on an ongoing basis down the line. > > > - More surface area means more work to keep things consistent. > Failure > > > to do that has, in fact, hurt the user experience. > > > - More surface area hurts agility. We want to do a few things > really > > > well as well as be agile to be able to build on our core > competency. > > > > > > > > > On Sat, Oct 1, 2016 at 5:38 AM Manikumar <manikumar.re...@gmail.com> > > > wrote: > > > > > > > Hi Jay, > > > > > > > > Thanks for your reply. > > > > > > > > I agree that we can not add all the clients/tools available in > ecosystem > > > > page to Kafka repo itself. But we feel REST Interface is different > from > > > > other clients/tools. Since any language that can work with HTTP can > > > > easily integrate with this interface, Having an "official" REST > > > > interface helps user community. This also helps us to integrate well > > > > with external management and provisioning tools. Apache Kafka > release > > > > with Java clients + REST interface is sufficient for most of the > user > > > > deployments/requirements. This helps users to deal with less number > > > > of distributions/builds. > > > > > > > > Thanks, > > > > Manikumar > > > > > > > > > > > > On Sat, Oct 1, 2016 at 4:24 AM, Jay Kreps <j...@confluent.io> wrote: > > > > > > > > > Hey guys, > > > > > > > > > > There's already a REST interface maintained as a separate > project--it's > > > > > open source and apache licensed and actively maintained ( > > > > > https://github.com/confluentinc/kafka-rest). What is wrong with > that? > > > > You > > > > > mentioned that there was some compatibility concern, but > compatibility > > > > has > > > > > to do with the consumer protocol guarantees not the repo the code > is > > > in, > > > > > right? Not sure that concern makes sense. > > > > > > > > > > We could argue for adding pretty much anything and everything in > the > > > > > ecosystem page in Kafka itself but I'm not sure that would make > the > > > > project > > > > > more agile. > > > > > > > > > > -Jay > > > > > > > > > > On Wed, Sep 28, 2016 at 12:04 AM, Manikumar > <manikumar.re...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > Hi Kafka Devs, > > > > > > > > > > > > I created KIP-80 to add Kafka REST Server to Kafka Repository. > > > > > > > > > > > > There are already open-source alternatives are available. But > we > > > would > > > > > > like to add REST server that > > > > > > many users ask for under Apache Kafka repo. Many data Infra > tools > > > comes > > > > > up > > > > > > with Rest Interface. > > > > > > It is useful to have inbuilt Rest API support for Produce, > Consume > > > > > messages > > > > > > and admin interface for > > > > > > integrating with external management and provisioning tools.This > will > > > > > also > > > > > > allow the maintenance of > > > > > > REST server and adding new features makes it easy because apache > > > > > community. > > > > > > > > > > > > The KIP wiki is the following: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > > > 80%3A+Kafka+Rest+Server > > > > > > > > > > > > Your comments and feedback are welcome. > > > > > > > > > > > > Thanks, > > > > > > Manikumar > > > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Neha > > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- This email, including attachments, is private and confidential. If you have received this email in error please notify the sender and delete it from your system. Emails are not secure and may contain viruses. No liability can be accepted for viruses that might be transferred by this email or any attachment. Any unauthorised copying of this message or unauthorised distribution and publication of the information contained herein are prohibited. 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB. Registered in England and Wales. Registered No. 04843573.
