I’m definitely looking forward to progress on this front. We’re currently running ZK only for Kafka. If we could have Kafka use our existing Etcd cluster, it would be one less critical piece of infrastructure to worry about, which would be great.
-- Daniel Nelson > On Aug 9, 2015, at 6:23 PM, Joe Stein <[email protected]> wrote: > > I have started writing a KIP about this topic > https://cwiki.apache.org/confluence/display/KAFKA/KIP-30+-+Allow+for+brokers+to+have+plug-able+consensus+and+meta+data+storage+sub+systems > with hopes to get more of it typed out and then circulate for discussion on > the dev list over the next ~ week. > > I think the plug-in that the project should support is the existing > implementation which will give folks not looking for an alternative a > stable upgrade path. > > There are 5 ways I have heard folks wanting to-do this. From the project > perspective I think that 1 (max 2) should be supported and the rest > available in contrib repo we can link to. > > Other thoughts/comments happy to catch those and update the KIP. > > Thanks! > > ~ Joe Stein > - - - - - - - - - - - - - - - - - - - > [image: Logo-Black.jpg] > http://www.elodina.net > http://www.stealth.ly > - - - - - - - - - - - - - - - - - - - > > On Sun, Aug 9, 2015 at 8:30 PM, Julio Castillo < > [email protected]> wrote: > >> The only reason for this request is because I may want to use alternatives >> like Consul. >> >> ** julio >> >> On 8/9/15, 3:40 PM, "Joe Lawson" <[email protected]> >> wrote: >> >>> Inline responses below. >>> >>> Sincerely, >>> >>> Joe Lawson >>> >>> On Aug 9, 2015 1:52 PM, "Julio Castillo" <[email protected]> >>> wrote: >>>> >>>> Thank for the lead. >>>> Does that mean that Kafka is/will be using Curator? >>> >>> I don't think so. >>> >>>> >>>> Also, this appears to simplify the interaction with ZooKeeper, but if I >>>> understand it correctly, it doesn易t abstract the interface where could >>>> plug-in a different service. >>> >>> You are right. I misunderstood what you meant and am unaware of any >>> ZooKeeper abstraction api that could allow other k/v stores underneath. >>> That is an interesting idea. Any reasons for the desire? >>> >>>> >>>> Thanks >>>> >>>> ** julio >>>> >>>> On 8/9/15, 10:21 AM, "Joe Lawson" <[email protected]> >>>> wrote: >>>> >>>>> Netflix contributed Curator >>>>> ( >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__curator.apache.org_&d >>>> >>>> =AwIBaQ&c=cKbMccWasSe6U4u_qE0M-qEjqwAh3shjuL5QPa1B7Yk&r=rJHFl4LhCQ-6kvKRO >>>> h >>>> >>>> IocflKqVSHRTvT-PgdZ5MFuS0&m=tR362iZgCAvRev2Bgf1Itdzh9j2bLPt9FEjoay6gO0A&s >>>> = >>>>> Rc_80-_j6gjBCp2GKL8sueIP8IKNdt7p7kgBhsPw2ZA&e= ) to Apache which >>>>> implements some generic zk recipes. >>>>> On Aug 9, 2015 11:39 AM, "Julio Castillo" >>>> <[email protected] >>>> >>>>> wrote: >>>>> >>>>>> Had there been any thought at abstracting the interface to ZooKeeper? >>>>>> >>>>>> The reason I'm asking is because I'm looking at Consul for service >>>>>> discovery today, perhaps a different one tomorrow, but the point here >>> is >>>>>> the ability to plug in any type of service discovery, K/V store >>> service. >>>>>> >>>>>> Any thoughts? >>>>>> >>>>>> ** julio >>>>>> >>>>>> NOTICE: This e-mail and any attachments to it may be privileged, >>>>>> confidential or contain trade secret information and is intended only >>>>>> for >>>>>> the use of the individual or entity to which it is addressed. If this >>>>>> e-mail was sent to you in error, please notify me immediately by >>>> either >>>>>> reply e-mail or by phone at 408.498.6000, and do not use, >>>> disseminate, >>>>>> retain, print or copy the e-mail or any attachment. All messages sent >>> to >>>>>> and from this e-mail address may be monitored as permitted by or >>>>>> necessary >>>>>> under applicable law and regulations. >>>>>> >>>> >>>> NOTICE: This e-mail and any attachments to it may be privileged, >>> confidential or contain trade secret information and is intended only for >>> the use of the individual or entity to which it is addressed. If this >>> e-mail was sent to you in error, please notify me immediately by either >>> reply e-mail or by phone at 408.498.6000, and do not use, disseminate, >>> retain, print or copy the e-mail or any attachment. All messages sent to >>> and from this e-mail address may be monitored as permitted by or necessary >>> under applicable law and regulations. >> >> >> NOTICE: This e-mail and any attachments to it may be privileged, >> confidential or contain trade secret information and is intended only for >> the use of the individual or entity to which it is addressed. If this >> e-mail was sent to you in error, please notify me immediately by either >> reply e-mail or by phone at 408.498.6000, and do not use, disseminate, >> retain, print or copy the e-mail or any attachment. All messages sent to >> and from this e-mail address may be monitored as permitted by or necessary >> under applicable law and regulations. >>
