Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-07 Thread Jay Kreps
Hey guys, I haven't thought this all the way through, but I think having metadata in the NetworkClient actually does make sense. After all it is the case that all requests are directed to Kafka nodes so this is a higher level of abstraction to work at. I think the feature you are looking for is to

Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-06 Thread Jiangjie Qin
Hey Guozhang, Good point about unify the handling of MetadataResponse and other ClientResponses. Thanks for all the feedbacks! I’ll take a shot and upload a patch for review. Jiangjie (Becket) Qin On 5/6/15, 2:19 PM, "Guozhang Wang" wrote: >Jiangjie, > >Just trying to figure the class referen

Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-06 Thread Guozhang Wang
Jiangjie, Just trying to figure the class reference hierarchy of this approach ("A <- B" means A either has a member variable of B or take B as API parameter). Metadata will have interface that takes in KafkaClient as a parameter, so Metadata <- KafkaClient 1. For producer: KafkaProducer <- Sen

Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-06 Thread Ewen Cheslack-Postava
On Wed, May 6, 2015 at 10:40 AM, Jiangjie Qin wrote: > Jun & Ewen, > > Thanks a lot for the comments. Both of them look better than my original > plan. > > Jun, one downside about not changing NetworkClient but use a different > metadata implementation is that NetworkClient will still have all th

Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-06 Thread Jiangjie Qin
Jun & Ewen, Thanks a lot for the comments. Both of them look better than my original plan. Jun, one downside about not changing NetworkClient but use a different metadata implementation is that NetworkClient will still have all the metadata related code there which makes it a little bit weird. I

Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-05 Thread Ewen Cheslack-Postava
+1 on trying to reuse the NetworkClient code. I think Jun's approach could work, but I'm wondering if refactoring a bit could get better separation of concerns without a somewhat awkward nop implementation of Metadata. I'm not sure what combination of delegation or subclassing makes sense yet, but

Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-05 Thread Jun Rao
Hi, Jiangjie, Thanks for taking on this. I was thinking that one way to decouple the dependency on Metadata in NetworkClient is the following. 1. Make Metadata an interface. 2. Rename current Metadata class to sth like KafkaMetadata that implements the Metadata interface. 3. Have a new NoOpMetada

[DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-05 Thread Jiangjie Qin
I am trying to see if we can reuse the NetworkClient class to be used in controller to broker communication. (Also, we can probably use KafkaConsumer which is already using NetworkClient in replica fetchers). Currently NetworkClient does the following things in addition to sending requests. 1