Re: How frequent does the follower replica issue a fetch request to leader replica?

[Yu Watanabe]

Hello Peter.

Thank you for the reply.

Now I understand.
I take a look at the " kafka.server,type=replica-fetcher-metrics" as well.

Thanks,
Yu Watanabe

On Wed, Apr 11, 2018 at 4:48 PM, Peter Bukowinski  wrote:

> Hi Yu,
>
> The broker property ‘replica.fetch.wait.max.ms’ determines the longest
> interval a follower should wait before issuing a fetch request of the
> leader. In reality these usually happen much more frequently, but it
> depends on the rate at which producers write to kafka topics (among other
> factors).
>
> The request rate of all your replica fetchers is one of the metrics that
> kafka reports. You can find it under 
> kafka.server,type=replica-fetcher-metrics.
> Many metrics are published for each leader and for each fetcher assigned to
> that leader (if you have more than one configured in the
> num.replica.fetchers broker parameter).
>
> For one of my small 5-broker clusters, my follower request rates range
> from 130 per second on the high end to 3 per second on the low end. I’ve
> configured 16 fetchers.
>
> --
> Peter Bukowinski
>
> > On Apr 10, 2018, at 10:54 PM, Yu Watanabe  wrote:
> >
> > Hello.
> >
> > I would like to ask question regarding to fetch request from follower to
> > leader replica.
> >
> > According to on line document it describes the flow of how fetch requests
> > are fetched for sync operation .
> >
> > https://kafka.apache.org/documentation/#replication
> >
> > "Followers consume messages from the leader just as a normal Kafka
> consumer
> > would and apply them to their own log.
> > Having the followers pull from the leader has the nice property of
> allowing
> > the follower to naturally batch together log entries they are applying to
> > their log."
> >
> > However,  I could not find the interval or frequency of the fetch
> request.
> > How often does follower replica issues fetch request to leader ?
> >
> > Thanks,
> > Yu
> >
> > --
> > Yu Watanabe
> > 渡辺 裕
> >
> > LinkedIn : jp.linkedin.com/in/yuwatanabe1
>



-- 
Yu Watanabe
渡辺 裕

LinkedIn : jp.linkedin.com/in/yuwatanabe1

Re: How frequent does the follower replica issue a fetch request to leader replica?

[Peter Bukowinski]

Hi Yu,

The broker property ‘replica.fetch.wait.max.ms’ determines the longest interval 
a follower should wait before issuing a fetch request of the leader. In reality 
these usually happen much more frequently, but it depends on the rate at which 
producers write to kafka topics (among other factors).

The request rate of all your replica fetchers is one of the metrics that kafka 
reports. You can find it under kafka.server,type=replica-fetcher-metrics. Many 
metrics are published for each leader and for each fetcher assigned to that 
leader (if you have more than one configured in the num.replica.fetchers broker 
parameter).

For one of my small 5-broker clusters, my follower request rates range from 130 
per second on the high end to 3 per second on the low end. I’ve configured 16 
fetchers.

--
Peter Bukowinski

> On Apr 10, 2018, at 10:54 PM, Yu Watanabe  wrote:
> 
> Hello.
> 
> I would like to ask question regarding to fetch request from follower to
> leader replica.
> 
> According to on line document it describes the flow of how fetch requests
> are fetched for sync operation .
> 
> https://kafka.apache.org/documentation/#replication
> 
> "Followers consume messages from the leader just as a normal Kafka consumer
> would and apply them to their own log.
> Having the followers pull from the leader has the nice property of allowing
> the follower to naturally batch together log entries they are applying to
> their log."
> 
> However,  I could not find the interval or frequency of the fetch request.
> How often does follower replica issues fetch request to leader ?
> 
> Thanks,
> Yu
> 
> -- 
> Yu Watanabe
> 渡辺 裕
> 
> LinkedIn : jp.linkedin.com/in/yuwatanabe1