Intra cluster replication is great and would alleviate (or probably eliminate) the need to have graceful decommission.
But that still does not answer the question: if one had to gracefully decommission a broker today in 0.7 (or in trunk or w/ patches), how would one do it? How can we make a broker read-only? Wouldn't there be a way to somehow bring consumers down, bring down the brokers that need to be decommissioned, then re-start those decommissioned brokers in another network where no producers are pushing any content into them, then restart the consumers inside of that other network to consume the stuff they had not consumed yet, and then move (stop and restart) the consumers back into their original network, where they could see and consume from the other brokers (which were not decommissioned and were still receiving stuff from the producers on that network). That seems awfully convoluted, and I'm not even sure it would work. Plus it would imply that there is some down time of the consumers (which matters for real-time oriented consumers, but less so for batch-oriented ones...). But still, I can't believe there wouldn't be any way at all... Is there a way to achieve something like that? -- Felix On Thu, Apr 5, 2012 at 9:17 PM, Bateman, Matt <mabate...@ebay.com> wrote: > Hi Jun, > > That would definitely solve the issue. I guess it's just a matter of > timing for 0.8... > > Thanks, > > Matt > > -----Original Message----- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Thursday, April 05, 2012 6:09 PM > To: kafka-users@incubator.apache.org > Subject: Re: Kafka 155 > > Matt, > > The main motivation for decommission is that one can let consumers drain > messages from a broker before taking it out. In 0.8, we are adding intra > cluster replication. So, a broker can be taken out anytime without > affecting consumers. Do you still see a need for decommission then? > > Thanks, > > Jun > > On Thu, Apr 5, 2012 at 2:13 PM, Bateman, Matt <mabate...@ebay.com> wrote: > > > Hi Guys, > > > > I see that 155 isn't targeted for a release. It deals with gracefully > > decommissioning a broker. Is there another way to achieve this? In > > other words, is there a way to prevent publishing to a broker but > > still allow consumers to pull messages until the broker is "empty"? > > > > Thanks, > > > > Matt > > > > >
