Hi, Ismael, Thanks for the pointers.
Hi, Jose, KIP-392 originally considered propagating HWM to followers. If there is a performance concern, we could focus only on the KRaft metadata topic. Also, KIP-392 implemented HWM propagation without adding the HWM field in the fetch request. Instead, the leader remembers the last propagated HWM to a follower. Is that easier to do in KRaft? If so, it's slightly better to avoid adding the HWM field in the fetch request since it's irrelevant to consumer clients. Thanks, Jun On Wed, Apr 30, 2025 at 2:22 PM Ismael Juma <m...@ismaeljuma.com> wrote: > And also https://issues.apache.org/jira/browse/KAFKA-9952 > > Ismael > > On Wed, Apr 30, 2025 at 2:18 PM Ismael Juma <m...@ismaeljuma.com> wrote: > > > Hi Jose, > > > > We did indeed run into a perf regression related to increased fetch > > request rate due to hw propagation to followers: > > > > https://issues.apache.org/jira/browse/KAFKA-9731 > > > > Ismael > > > > On Wed, Apr 30, 2025 at 12:34 PM José Armando García Sancio > > <jsan...@confluent.io.invalid> wrote: > > > >> Hi Alyssa, > >> > >> On Wed, Apr 30, 2025 at 12:52 PM Alyssa Huang > >> <ahu...@confluent.io.invalid> wrote: > >> > Is the intent to say that in addition to all the current conditions, > >> > the HWM check is an additional condition that *must *be met in order > for > >> > the fetch request to be parked? Just confirming since I'm not sure if > I > >> > should interpret "may also park" too literally. > >> > >> It is an additional condition that may be considered. The difficulty > >> and my hesitation in being prescriptive in the implementation is that > >> completing the FETCH request because of the HWM may impact the > >> utilization and throughput of Kafka. In many cases there is a > >> trade-off between lower latencies and higher throughput. > >> > >> My position is that the remote HWM can be used when handling FETCH > >> requests but the exact handling and implementation may change. > >> > >> Thanks, > >> -- > >> -José > >> > > >
