Re: Scaling up kafka consumer applications(kafka streams)

2018-04-11 Thread Shivam Sharma
No I am not repartitioning data later. But I wanted to run 10 applications. Is there any way to do this? On Mon, Apr 9, 2018 at 1:44 AM, Matthias J. Sax wrote: > It depend on the concrete program you write... Most likely it's 4, but > hard to say without more information.

Re: How frequent does the follower replica issue a fetch request to leader replica?

2018-04-11 Thread Peter Bukowinski
Hi Yu, The broker property ‘replica.fetch.wait.max.ms’ determines the longest interval a follower should wait before issuing a fetch request of the leader. In reality these usually happen much more frequently, but it depends on the rate at which producers write to kafka topics (among other

Re: How frequent does the follower replica issue a fetch request to leader replica?

2018-04-11 Thread Yu Watanabe
Hello Peter. Thank you for the reply. Now I understand. I take a look at the " kafka.server,type=replica-fetcher-metrics" as well. Thanks, Yu Watanabe On Wed, Apr 11, 2018 at 4:48 PM, Peter Bukowinski wrote: > Hi Yu, > > The broker property ‘replica.fetch.wait.max.ms’

Re: Scaling up kafka consumer applications(kafka streams)

2018-04-11 Thread Matthias J. Sax
You might need to pipe the data through a topic with 10 partitions using an explicit call to `.through()` -Matthias On 4/11/18 12:48 AM, Shivam Sharma wrote: > No I am not repartitioning data later. But I wanted to run 10 applications. > Is there any way to do this? > > On Mon, Apr 9, 2018 at