Trunk is newer than the 0.7.0 jar. During rebalance, dups are introduced per partition. So, the more the # of partitions, the more dups.
Jun On Wed, May 16, 2012 at 11:45 PM, navneet sharma < navneetsharma0...@gmail.com> wrote: > I downloaded the tar from the download link provided in quickstart page. > Almost more than a month back. > > I trunk maintaining different code than the tar? > > Can number of partitions cause this problem, beacuse i am using 2 > partitions on each of the two brokers.? > > Thanks, > Navneet Sharma > > On Wed, May 16, 2012 at 10:00 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > Technically this is the guarantee we provide--at least once delivery. > > It is very expensive to completely eliminate this possibility in the > > general case as you need to co-ordinate any state changes the consumer > > makes with committing the offset that marks the position. But we have > > improved the common cases for normal rebalancing so if you are using > > trunk the only time this would happen is when there is a hard crash of > > a process. > > > > -Jay > > > > On Wed, May 16, 2012 at 2:41 AM, navneet sharma > > <navneetsharma0...@gmail.com> wrote: > > > Hi, > > > > > > I tried a scenario wherein: > > > 1) i had 1 producer and 3 consumers subscribed for a topic - > "cartTopic", > > > all in same group. > > > 2) Now, when everything is executing, i introduced another consumer for > > the > > > same topic and in the same group. So, overall there are 4 consumers. > > > 3) Ofcourse, it triggered re-balancing. > > > > > > But then final result is that few messages are duplicated. > > > In my example run, producer sent 800,000 records, but consumer received > > > 801,448 records. > > > I am using log4j to generate the output file. > > > > > > Is there any reasons for duplicacy? > > > > > > Thanks, > > > Navneet Sharma > > >
