>> 2. One of the streams times out (or has some other error) and it's iterator is now in mode FAILED
This seems like a code bug. We have fixed quite a bunch of bugs in the consumer since 0.6. Can you reproduce this with trunk or the 0.7 branch ? Thanks, Neha On Thu, Dec 22, 2011 at 1:40 AM, David Ross <dyr...@klout.com> wrote: > Howdy, > > I have come across a potential issue with the Consumer API in Kafka 0.6. > After some tests and reading through the code, I believe the following > scenario could result in an unhappy situation: > > 1. Create 5 streams listening to a topic > 2. One of the streams times out (or has some other error) and it's > iterator is now in mode FAILED > 3. (I think) Messages will now come into all five streams, but one of > them won't yield any new messages because of (2) > > The only solution I can think of is if an error is detected in any of the > streams, all of them should be stopped and a new set of streams should be > made. This seems unfavorable and would require messy concurrent programming. > > So is this an actual problem? Is there a better workaround? Have I > misinterpreted something? > > > Thanks, > > David
