Hi, Jiunn-Yang, JR2. We should try to maintain consistency across our APIs. So, it would be useful to understand if the user-facing part of this KIP is what share consumer wants too. It would be problematic if we proceed with this approach only for the share consumer to choose a different one later.
JR3. I was thinking that if we add the NewPartitions field in GroupHeartbeatResponse, there is no need to expose the group/partition creation timestamp to other RPCs and the Cli. JR5. Since this changes the on-disk data format for PartitionRecord, we need to gate this by a new MV. It also changes the format of ConsumerGroupMetadataValue, so we need to gate it too. I'm not sure if this requires a new MV or a new GV. Thanks, Jun On Thu, Jun 25, 2026 at 5:52 AM 黃竣陽 <[email protected]> wrote: > Hello Jun, chia, > > Thanks for your feedback. > > JR3 JR4 I have updated the KIP. > > JR2 The new-partition detection mechanism is reusable. The coordinator > already has > access to both timestamps required for classification, so the > determination can be made > server-side without introducing any new RPCs. The main difference is where > the policy is > enforced: modern consumer resolves it on the client, whereas a share group > resolves the > starting offset on the partition leader. As a result, the classification > decision would need to > be propagated through the existing share-state initialization path. We > would also need to persist > the group's creation time as part of the share-group state, since that > information is not stored today. > > None of this is a blocker, but it does require changes to the share-state > format and introduces > a share-specific configuration. Given that additional scope, I would > prefer to keep share groups out > of scope for this KIP and address them in a follow-up effort, while > documenting the relevant reuse > opportunities here. > > Best Regards, > Jiunn-Yang > > > Chia-Ping Tsai <[email protected]> 於 2026年6月25日 下午1:31 寫道： > > > >> It inspires me to think about mirror clusters. What if the mirror group > is created 'after' the expanded partitions? In this scenario, the new > configuration 'max.age.ms' could handle it if users are aware of the > migration and set a suitable time interval. Of course, this would require > both creation times to be exposed to clients. > > > > Thinking more about the mirror cluster scenario we discussed, I realize > it introduces significant complexity. Handling it properly would require a > tolerance time window rather than a strict time interval to account for > replication lag. To prevent scope creep and keep KIP-1327 focused on its > primary goal, I think it would be best to defer the multi-cluster/mirroring > scenario and explore those approaches in a separate KIP > > > > On 2026/06/24 21:29:19 Chia-Ping Tsai wrote: > >> hi all, > >> > >>> JR3. The KIP exposes the group/partition creation time to the client. > I am > >>> not sure if there are other use cases for those timestamps. An > alternative > >>> is to avoid exposing those timestamps to the client. The group > coordinator > >>> will categorize the partitions using those timestamps and include the > >>> categorization in the assigned partitions in GroupHeartbeatResponse. > >> > >> It inspires me to think about mirror clusters. What if the mirror group > is created 'after' the expanded partitions? In this scenario, the new > configuration 'max.age.ms' could handle it if users are aware of the > migration and set a suitable time interval. Of course, this would require > both creation times to be exposed to clients. > >> > >> Best, > >> Chia-Ping > >> > >> On 2026/06/24 20:46:16 Jun Rao via dev wrote: > >>> Hi, Jiunn-Yang, > >>> > >>> Thanks for the updated KIP. > >>> > >>> JR2. Share consumer (KIP-932) states: "If the number of partitions is > >>> increased for a topic with a subscription in a share group, the SPSO > for > >>> the newly created share-partitions is initialized to 0 ". However, this > >>> part has not been implemented yet. Currently, all partitions without > offset > >>> follows the strategy given by share.auto.offset.reset. So, we have an > >>> opportunity to see if it's better to align this KIP's changes with the > >>> share consumer. > >>> JR2.1 The first part is whether customizing the offset for new > partitions > >>> to something other than 0 is useful in the share consumer. It seems > that > >>> the motivation in this KIP applies to share consumer too. > >>> JR2.2 The second part is how new partitions are detected. The mechanism > >>> introduced in this KIP seems quite effective. A group coordinator could > >>> pick up newly created partitions for an existing topic or newly created > >>> topics matching the regex. Those partitions are categorized as new > >>> partitions and will follow the strategy given by > >>> auto.offset.reset.new.partitions. > >>> This makes sense since those partitions typically don't have a large > >>> backlog. A consumer could also additionally subscribe to an existing > topic > >>> in an existing group. Those partitions will be categorized as existing > >>> partitions. This also makes sense since those partitions could have a > large > >>> backlog and it is better to follow the strategy given by > auto.offset.reset. It > >>> will be useful to think through whether the logic for categorizing new > >>> partitions should also be reused in the share consumer. > >>> > >>> JR3. The KIP exposes the group/partition creation time to the client. > I am > >>> not sure if there are other use cases for those timestamps. An > alternative > >>> is to avoid exposing those timestamps to the client. The group > coordinator > >>> will categorize the partitions using those timestamps and include the > >>> categorization in the assigned partitions in GroupHeartbeatResponse. > >>> > >>> JR4. It would be useful to support this feature on existing groups as > well. > >>> For example, any partition without a creation timestamp can always be > >>> categorized as an existing partition. > >>> > >>> Jun > >>> > >>> On Wed, Jun 24, 2026 at 6:12 AM 黃竣陽 <[email protected]> wrote: > >>> > >>>> Hello Andrew, > >>>> > >>>> Thanks for your feedback, > >>>>>> I've just been re-reading this KIP to ensure that share group > support is > >>>>>> not needed. I've discovered that we made a mistake implementing > KIP-932 > >>>> in > >>>>>> this area, so we'll rectify that and check whether it's really tight > >>>> enough > >>>>>> not to require KIP-1327. > >>>> > >>>> Could you walk us through the specific area where the mistake was > found? > >>>> That would > >>>> help us assess whether it naturally fits within the scope of KIP-1327 > or > >>>> should be addressed > >>>> separately. > >>>> > >>>> AS1 I have updated the KIP > >>>> > >>>> Best Regards, > >>>> Jiunn-Yang > >>>> > >>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月24日 晚上8:49 寫道： > >>>>> > >>>>> hi Andrew > >>>>> > >>>>>> I've discovered that we made a mistake implementing KIP-932 in this > >>>> area, > >>>>> so we'll rectify that and check whether it's really tight enough not > to > >>>>> require KIP-1327. > >>>>> > >>>>> Would you mind sharing more details about that? If it's not anything > >>>> close > >>>>> to NP-hard, we could definitely look into covering it within KIP-1327 > >>>>> > >>>>> Best, > >>>>> Chia-Ping > >>>>> > >>>>> > >>>>> Andrew Schofield <[email protected]> 於 2026年6月24日週三 下午7:49寫道： > >>>>> > >>>>>> Hi Jiunn-Yang, > >>>>>> I've just been re-reading this KIP to ensure that share group > support is > >>>>>> not needed. I've discovered that we made a mistake implementing > KIP-932 > >>>> in > >>>>>> this area, so we'll rectify that and check whether it's really tight > >>>> enough > >>>>>> not to require KIP-1327. > >>>>>> > >>>>>> One trivial comments for consistency. > >>>>>> > >>>>>> AS1: In kafka-consumer-groups.sh, unknown column data is > represented by > >>>>>> "-", not "N/A". In kafka-topics.sh, we do use "N/A". > >>>>>> > >>>>>> Thanks, > >>>>>> Andrew > >>>>>> > >>>>>> On 2026/06/23 23:23:38 黃竣陽 wrote: > >>>>>>> Hi all, > >>>>>>> > >>>>>>> Manually bumping this thread. > >>>>>>> > >>>>>>> Best Regards, > >>>>>>> Jiunn-Yang > >>>>>>> > >>>>>>>> 黃竣陽 <[email protected]> 於 2026年6月17日 晚上9:17 寫道： > >>>>>>>> > >>>>>>>> Hello chia, > >>>>>>>> > >>>>>>>> Thanks for the feedback, I have updated the KIP. > >>>>>>>> > >>>>>>>> Best Regards, > >>>>>>>> Jiunn-Yang > >>>>>>>> > >>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月17日 凌晨12:47 寫道： > >>>>>>>>> > >>>>>>>>> hi Jiunn-Yang > >>>>>>>>> > >>>>>>>>>> When the config is set on a cluster that has not yet been > >>>>>> upgraded... classification cannot occur... the consumer falls back > to > >>>> the > >>>>>> base auto.offset.reset for the affected partitions. No exception is > >>>> thrown, > >>>>>> and no operational disruption results. > >>>>>>>>> > >>>>>>>>> Existing group can't take advantage of this excellent new > >>>>>> configuration. Allowing users to modify the group creation time > might be > >>>>>> overkill. Instead, we could print a useful warning message to guide > >>>> users. > >>>>>> For example, we can suggest that they re-create the group with their > >>>>>> existing committed offsets > >>>>>>>>> > >>>>>>>>>> Protocol changes > >>>>>>>>> > >>>>>>>>> Would you mind listing those RPC changes in a table format? > >>>>>>>>> > >>>>>>>>>> The full interaction matrix between the base policy and the > >>>>>> new-partition policy is: > >>>>>>>>> > >>>>>>>>> Please add a filed to describe the target scenario when using > these > >>>>>> policies > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Chia-Ping > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 2026/06/16 16:14:49 黃竣陽 wrote: > >>>>>>>>>> Hello Jun, chia, > >>>>>>>>>> > >>>>>>>>>> Thanks for the feedback, I have updated the KIP for the new > >>>>>>>>>> approach, PTAL > >>>>>>>>>> > >>>>>>>>>> Best Regards, > >>>>>>>>>> Jiunn-Yang > >>>>>>>>>> > >>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月16日 上午8:23 寫道： > >>>>>>>>>>> > >>>>>>>>>>> hi Jun > >>>>>>>>>>> > >>>>>>>>>>> Yes, your approach is great. I think the combination of latest > (for > >>>>>> existing partitions) and by_duration (for new partitions) can > address > >>>> 99% > >>>>>> of the complaints I have heard regarding this issue. > >>>>>>>>>>> > >>>>>>>>>>> Also, leveraging the group creation time here opens the door to > >>>>>> implementing a new policy based on timestamp seek in the future, > should > >>>> the > >>>>>> community want to pursue that. > >>>>>>>>>>> > >>>>>>>>>>> Thanks for your patience and constructive feedback. We will > update > >>>>>> the KIP accordingly. > >>>>>>>>>>> > >>>>>>>>>>> Best, Chia-Ping > >>>>>>>>>>> > >>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月16日 清晨5:11 > 寫道： > >>>>>>>>>>>> > >>>>>>>>>>>> Hi, Chia-Ping, > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks for the reply. > >>>>>>>>>>>> > >>>>>>>>>>>> I agree that it's probably useful to allow a user to > configure a > >>>>>> different > >>>>>>>>>>>> offset policy for existing partitions vs new partitions. > However, > >>>>>> using > >>>>>>>>>>>> group creation time to capture that seems more intuitive. > Here is > >>>>>> another > >>>>>>>>>>>> proposal: remove auto.offset.reset.max.age.ms and categorize > new > >>>>>> partitions > >>>>>>>>>>>> based on group creation time. Introduce > >>>>>>>>>>>> a new config auto.offset.reset.new.partitions whose values > can be > >>>>>> earliest, > >>>>>>>>>>>> latest and by_duration, the same as auto.offset.reset. Users > can > >>>>>> set > >>>>>>>>>>>> `auto.offset.reset.new.partitions` to `earliest` if they > want to > >>>>>> guarantee > >>>>>>>>>>>> no data loss on new partitions. They can also use by_duration > to > >>>>>> set an > >>>>>>>>>>>> upper bound on the backlog replayed, which can be different > from > >>>>>> that of > >>>>>>>>>>>> the existing partitions. This will address your concern about > too > >>>>>> much > >>>>>>>>>>>> backlog being replayed when the offsets are lost. What do you > >>>>>> think? > >>>>>>>>>>>> > >>>>>>>>>>>> Jun > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> On Mon, Jun 15, 2026 at 10:39 AM Chia-Ping Tsai < > >>>>>> [email protected]> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> hi Jun > >>>>>>>>>>>>> > >>>>>>>>>>>>> The most important part of this story is how users should > expect > >>>>>> the data > >>>>>>>>>>>>> they can see when using the latest or by_duration policy with > >>>>>> expanded > >>>>>>>>>>>>> partitions. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Yes, the by_duration policy can minimize data loss, but it is > >>>>>>>>>>>>> non-deterministic, which means users will either read too > many > >>>>>> historical > >>>>>>>>>>>>> records from existing partitions or lose some records from > >>>>>> expanded > >>>>>>>>>>>>> partitions. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Also, I agree that auto.offset.reset.max.age.ms > >>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > >>>>>>> > >>>>>>>>>>>>> is a bit hard to understand, and that is why I preferred > having a > >>>>>> whole new > >>>>>>>>>>>>> policy based entirely on group creation time (KIP-1282) > >>>>>>>>>>>>> > >>>>>>>>>>>>> Best, > >>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>> > >>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月16日週二 > 上午1:08寫道： > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi, Chia-Ping and Jiunn-Yang, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks for the reply. I am still trying to understand the > value > >>>>>> of the new > >>>>>>>>>>>>>> configs with the KIP. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> The motivation of the KIP is that a user doesn't want to > miss > >>>>>> the data if > >>>>>>>>>>>>>> the backlog is small. The backlog of the existing partition > is > >>>>>> easy to > >>>>>>>>>>>>>> understand because it relates to retention time. The > backlog for > >>>>>> the new > >>>>>>>>>>>>>> partition is a bit subtle to understand since it depends on > the > >>>>>> metadata > >>>>>>>>>>>>>> refresh delay. To set auto.offset.reset.max.age.ms > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > >>>>>>> , > >>>>>>>>>>>>>> the user needs to > >>>>>>>>>>>>>> understand the metadata refresh delay on the consumer side > and > >>>>>> use it to > >>>>>>>>>>>>>> set the config. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Now, let's consider the alternative: setting the same value > for > >>>>>> the > >>>>>>>>>>>>>> existing by_duration policy. The KIP lists three issues with > >>>> this > >>>>>>>>>>>>>> approach. > >>>>>>>>>>>>>> 1. It computes the seek target client-side as now() - > duration, > >>>>>> which > >>>>>>>>>>>>>> introduces clock skew across consumers and forces operators > to > >>>>>> choose > >>>>>>>>>>>>>> overly large durations, causing unnecessary reprocessing. > >>>>>>>>>>>>>> 2. The target timestamp is recomputed on each retry, so > failed > >>>>>>>>>>>>>> ListOffsetsRequest retries can shift the target forward and > >>>>>> potentially > >>>>>>>>>>>>>> miss records produced between attempts. > >>>>>>>>>>>>>> 3. It applies uniformly to all partitions without committed > >>>>>> offsets, and > >>>>>>>>>>>>>> cannot distinguish newly expanded partitions from > long-existing > >>>>>> partitions > >>>>>>>>>>>>>> newly assigned to the group, leading to unnecessary replay. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Issues 1 and 2 are uncommon and can be mitigated by adding > a bit > >>>>>> buffer to > >>>>>>>>>>>>>> the metadata refresh delay. We could also consider > improving the > >>>>>>>>>>>>>> implementation. For issue 3, the metadata refresh delay is > >>>>>> typically low > >>>>>>>>>>>>>> (in the order of minutes with the classic consumer and tens > of > >>>>>> seconds > >>>>>>>>>>>>>> with > >>>>>>>>>>>>>> the new consumer). If a user is ok with reading that much > >>>>>> backlog for new > >>>>>>>>>>>>>> partitions, it seems they will be ok doing the same for > existing > >>>>>>>>>>>>>> partitions. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> So, instead of introducing a new config, could we just > reuse the > >>>>>> existing > >>>>>>>>>>>>>> config with better documentation and/or implementation? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Sat, Jun 13, 2026 at 12:19 AM 黃竣陽 <[email protected]> > >>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hello Jun, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> You're right that group creation time is the more intuitive > >>>>>> answer at > >>>>>>>>>>>>>>> first glance, > >>>>>>>>>>>>>>> the KIP's own motivation talks about partitions that > "predate > >>>>>> the group" > >>>>>>>>>>>>>>> vs partitions > >>>>>>>>>>>>>>> "created during group runtime," which directly points to a > >>>>>>>>>>>>>> group-lifecycle > >>>>>>>>>>>>>>> classifier. > >>>>>>>>>>>>>>> I'd like to walk through why we landed on partition age, > and > >>>> the > >>>>>>>>>>>>>>> trade-offs we considered. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We evaluated three candidate signals: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 1. `by_duration:5secs` > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> This covers the metadata blindness window, but has issues > the > >>>>>> KIP > >>>>>>>>>>>>>>> currently documents > >>>>>>>>>>>>>>> under "Why not use `by_duration`?": > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> - Client-side `now() - duration` introduces clock skew > across > >>>>>> consumers. > >>>>>>>>>>>>>>> - `ListOffsets` retries shift the target forward, > potentially > >>>>>> missing > >>>>>>>>>>>>>>> records produced between > >>>>>>>>>>>>>>> attempts. > >>>>>>>>>>>>>>> - It applies uniformly to all partitions without committed > >>>>>> offsets, > >>>>>>>>>>>>>>> including pre-existing partitions > >>>>>>>>>>>>>>> newly assigned to the group, causing unnecessary replay. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2. Group creation time as classifier > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> This works cleanly when the consumer is actively running. > Our > >>>>>> concern > >>>>>>>>>>>>>>> is the idle / late-rejoin case: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> T=0: Group created. > >>>>>>>>>>>>>>> T=1..T=100: Consumer idle (down, disconnected, etc.). > >>>>>>>>>>>>>>> T=50: Partition added during the idle window. > >>>>>>>>>>>>>>> T=100: Consumer resumes. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Under group creation time, the new partition is classified > as > >>>>>> new > >>>>>>>>>>>>>>> (`50 > 0`) and reset to `earliest`, replaying everything > from > >>>>>> T=50. > >>>>>>>>>>>>>>> But during `[T=1, T=100]`, base partitions also accumulated > >>>>>> data that > >>>>>>>>>>>>>>> the consumer accepts as lost — that is precisely the > contract > >>>> of > >>>>>>>>>>>>>>> `auto.offset.reset=latest`. There is no principled reason > to > >>>>>> treat > >>>>>>>>>>>>>>> the new partition differently; both contain backlog > accumulated > >>>>>> during > >>>>>>>>>>>>>>> the same idle window. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> This aligns with the "backlog is backlog” principle you > raised > >>>>>> in > >>>>>>>>>>>>>>> the KIP-1282 thread: a `latest` user has tolerated some > backlog > >>>>>> on > >>>>>>>>>>>>>>> every other partition during the same idle period; forcing > >>>>>> 0-backlog > >>>>>>>>>>>>>>> tolerance only on new partitions would be inconsistent with > >>>> that > >>>>>>>>>>>>>>> tolerance. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 3. Partition age vs threshold > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Partition age corresponds to the actual silent data loss > >>>> window, > >>>>>>>>>>>>>>> the gap between partition creation and the consumer’s > metadata > >>>>>>>>>>>>>>> refresh. Within this window, data loss is genuinely > silent: the > >>>>>>>>>>>>>>> consumer had no opportunity to know about the partition. > >>>>>> Outside this > >>>>>>>>>>>>>>> window, missing data reflects either: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> - (a) the user’s tolerated cost of running with idle > consumers, > >>>>>> or > >>>>>>>>>>>>>>> - (b) an operational issue to surface via monitoring, not > via > >>>>>> reset > >>>>>>>>>>>>>> policy. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We did not choose partition age because it is more elegant > than > >>>>>> group > >>>>>>>>>>>>>>> creation time — we chose it because its failure mode > (requires > >>>> a > >>>>>>>>>>>>>>> threshold) is > >>>>>>>>>>>>>>> less invasive than the failure mode of group creation time > >>>>>> (overrides > >>>>>>>>>>>>>>> user-stated > >>>>>>>>>>>>>>> `latest` intent during idle periods). > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月13日 上午11:52 > 寫道： > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi Jun, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Relying on both creation times will create an inconsistent > >>>>>> scenario. A > >>>>>>>>>>>>>>>> consumer that lost all offsets due to a long sleep will > seek > >>>>>> to the > >>>>>>>>>>>>>>>> beginning for the partitions created later than the group. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> That is why we initially proposed KIP-1282 to fix the > >>>>>> inconsistency > >>>>>>>>>>>>>>> using a > >>>>>>>>>>>>>>>> whole new policy. Since KIP-1282 couldn't reach a > consensus, > >>>>>> KIP-1327 > >>>>>>>>>>>>>>> goes > >>>>>>>>>>>>>>>> back to using flexible configurations to prevent users > from > >>>>>> falling > >>>>>>>>>>>>>> into > >>>>>>>>>>>>>>>> that pitfall. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Best, Chia-Ping > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月13日週六 > >>>>>> 上午6:49寫道： > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks for the reply and sorry for the late reply. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> JR1. The design of auto.offset.reset.max.age.ms > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > >>>>>>> > >>>>>>>>>>>>>> still feels weird to > >>>>>>>>>>>>>>> me. > >>>>>>>>>>>>>>>>> It > >>>>>>>>>>>>>>>>> categorizes partitions as new or existing based on the > >>>>>> partition > >>>>>>>>>>>>>>> creation > >>>>>>>>>>>>>>>>> time. Intuitively, the categorization should be based on > the > >>>>>> group > >>>>>>>>>>>>>>> creation > >>>>>>>>>>>>>>>>> time: all partitions existing when the group is created > are > >>>>>> existing > >>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>> all partitions created after the group creation are new > >>>>>> partitions. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Tue, Jun 9, 2026 at 8:51 AM 黃竣陽 <[email protected]> > >>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Manually bumping this thread. If there is no further > >>>>>>>>>>>>>>>>>> discussion, I will close the vote. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年6月1日 晚上7:16 寫道： > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Hello Jian, > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Thanks for your feedback, > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Agreed, partition expansion is a common operational > task, > >>>>>> not an > >>>>>>>>>>>>>> edge > >>>>>>>>>>>>>>>>>>> case. I've updated the Motivation section accordingly. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> jian fu <[email protected]> 於 2026年6月1日 下午5:49 寫道： > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Hi Jiunn-Yang: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks for the KIP. I think it would be useful to > clarify > >>>>>> that > >>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>> is a > >>>>>>>>>>>>>>>>>>>> common scenario rather than an edge case, which > further > >>>>>>>>>>>>>> demonstrates > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>> need for this optimization. For example: > >>>>>>>>>>>>>>>>>>>> A partition expansion is a common operational task in > >>>>>> Kafka: To > >>>>>>>>>>>>>>>>> balance > >>>>>>>>>>>>>>>>>>>> resource utilization and cost, topics are typically > >>>>>> created with a > >>>>>>>>>>>>>>>>>> moderate > >>>>>>>>>>>>>>>>>>>> default partition count. However, as traffic grows > over > >>>>>> time, it > >>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>> often > >>>>>>>>>>>>>>>>>>>> necessary to increase the number of partitions to > >>>>>> accommodate the > >>>>>>>>>>>>>>>>> higher > >>>>>>>>>>>>>>>>>>>> workload. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Regards > >>>>>>>>>>>>>>>>>>>> Jian > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 于2026年5月30日周六 22:31写道： > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Hello chia, > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Thanks for the comments, I have updated the KIP! > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年5月30日 > 晚上8:29 > >>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Hi Jiunn-Yang, > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Would you mind removing the terms "hot" and "cold" > when > >>>>>>>>>>>>>> describing > >>>>>>>>>>>>>>>>>>>>>> partitions in the KIP? I understand you are using > them > >>>> to > >>>>>>>>>>>>>> describe > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>> "freshness" or the users' need for the records, but > >>>>>> applying > >>>>>>>>>>>>>> these > >>>>>>>>>>>>>>>>>> terms > >>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>> the partition itself feels a bit unnatural. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> After all, in this scenario, users don't really care > >>>>>> whether a > >>>>>>>>>>>>>>>>>> partition > >>>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>> newly expanded or not. Their only expectation is > that > >>>>>> they won't > >>>>>>>>>>>>>>>>>> silently > >>>>>>>>>>>>>>>>>>>>>> lose any live records produced to the topic during > their > >>>>>> active > >>>>>>>>>>>>>>>>>>>>> consumption. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Best, Chia-Ping > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月30日週六 下午12:30寫道： > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Hello Jun, > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback, I have updated the KIP > >>>>>> motivation > >>>>>>>>>>>>>>> section. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 > 2026年5月30日 > >>>>>> 凌晨1:12 > >>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Thanks for the reply. I think we need a stronger > >>>>>> motivation > >>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> KIP. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> The KIP says "The core insight is that not all > >>>>>> partitions > >>>>>>>>>>>>>> without > >>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>> committed offset are the same. A newly expanded > >>>>>> partition > >>>>>>>>>>>>>> (hot) > >>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>> fundamentally different from a partition the > consumer > >>>>>> has > >>>>>>>>>>>>>> never > >>>>>>>>>>>>>>>>> seen > >>>>>>>>>>>>>>>>>>>>>>>> because it predates the group (cold)." Why is the > hot > >>>>>>>>>>>>>> partition > >>>>>>>>>>>>>>>>>>>>>>>> fundamentally different from the cold? > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> The KIP says "The existing by_duration policy is > also > >>>>>>>>>>>>>>> insufficient > >>>>>>>>>>>>>>>>>>>>>>> because: > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> - The calculated seek time (now() - duration) > varies > >>>>>> across > >>>>>>>>>>>>>> nodes > >>>>>>>>>>>>>>>>>> due > >>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>> clock skew. To be safe, users must set an overly > large > >>>>>>>>>>>>>> duration, > >>>>>>>>>>>>>>>>>>>>>>> causing > >>>>>>>>>>>>>>>>>>>>>>>> unnecessary reprocessing. > >>>>>>>>>>>>>>>>>>>>>>>> - On network errors, the client recalculates the > seek > >>>>>> time on > >>>>>>>>>>>>>>>>> retry, > >>>>>>>>>>>>>>>>>>>>>>>> shifting the target timestamp forward and risking > data > >>>>>> loss." > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> However, both of these situations are rare. If > these > >>>>>> issues > >>>>>>>>>>>>>>>>> persist, > >>>>>>>>>>>>>>>>>>>>> more > >>>>>>>>>>>>>>>>>>>>>>>> severe problems likely exist elsewhere. Rare > >>>>>> situations don't > >>>>>>>>>>>>>>>>> need a > >>>>>>>>>>>>>>>>>>>>>>> common > >>>>>>>>>>>>>>>>>>>>>>>> solution. If users care about those rare > situations, > >>>>>> they can > >>>>>>>>>>>>>>>>>> implement > >>>>>>>>>>>>>>>>>>>>>>>> customized logic using > >>>>>>>>>>>>>>>>>>>>> ConsumerRebalanceListener.onPartitionsAssigned(). > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> On Sun, May 17, 2026 at 6:50 AM 黃竣陽 < > >>>>>> [email protected]> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Hello chia, > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback, > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> If the creation time exists, the returned value > >>>>>> should > >>>>>>>>>>>>>> always > >>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>> greater > >>>>>>>>>>>>>>>>>>>>>>>>> than or equal to zero, right? > >>>>>>>>>>>>>>>>>>>>>>>>> I have explicitly mentioned this in the KIP. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive > >>>>>> any > >>>>>>>>>>>>>>>>>> field > >>>>>>>>>>>>>>>>>>>>>>>>> absent UnsupportedVersionException > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> The earliest point at which we can detect the > version > >>>>>>>>>>>>>> mismatch > >>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>> during > >>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>> first metadata fetch after assignment, which > occurs > >>>>>> inside > >>>>>>>>>>>>>>>>> poll(). > >>>>>>>>>>>>>>>>>>>>>>>>> Therefore, the > >>>>>>>>>>>>>>>>>>>>>>>>> user would encounter an > UnsupportedVersionException > >>>>>> from > >>>>>>>>>>>>>> poll(). > >>>>>>>>>>>>>>>>>> I’ll > >>>>>>>>>>>>>>>>>>>>>>>>> clarify this in the KIP. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > 2026年5月17日 > >>>>>> 下午4:50 > >>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> hi Jiunn > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> PartitionAgeMs (int64, default -1): The age of > this > >>>>>>>>>>>>>> partition > >>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>>>>> milliseconds, computed server-side by the broker > as > >>>>>>>>>>>>>>>>>>>>> broker_current_time > >>>>>>>>>>>>>>>>>>>>>>> - > >>>>>>>>>>>>>>>>>>>>>>>>> partition_creation_time. Returns -1 if the broker > >>>>>> does not > >>>>>>>>>>>>>>>>> support > >>>>>>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>>>>>> feature or the partition creation time is > unknown. > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> If the creation time exists, the returned value > >>>>>> should > >>>>>>>>>>>>>> always > >>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>> greater > >>>>>>>>>>>>>>>>>>>>>>>>> than or equal to zero, right? > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive > >>>>>> any > >>>>>>>>>>>>>>>>>> field > >>>>>>>>>>>>>>>>>>>>>>>>> absent UnsupportedVersionException > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Will user encounter UnsupportedVersionException > when > >>>>>> calling > >>>>>>>>>>>>>>>>>>>>> `poll()`? > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/16 04:30:49 黃竣陽 wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>> Hello Jun, chia, > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I've updated KIP-1327 with a design change > based on > >>>>>> the > >>>>>>>>>>>>>>>>>> discussion > >>>>>>>>>>>>>>>>>>>>>>>>>>> feedback. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> The updated design decouples the new-partition > >>>> reset > >>>>>>>>>>>>>> behavior > >>>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>>>>>>>> the base auto.offset.reset policy: > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> - auto.offset.reset.max.age.ms > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > >>>>>>> > >>>>>>>>>>>>>> now applies to all > >>>>>>>>>>>>>>>>>> auto.offset.reset > >>>>>>>>>>>>>>>>>>>>>>>>> values > >>>>>>>>>>>>>>>>>>>>>>>>>>> (latest, earliest, by_duration, none). > >>>>>>>>>>>>>>>>>>>>>>>>>>> - For new ("hot") partitions, the consumer > resets > >>>> to > >>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> > >>>>>>>>>>>>>> .partitions > >>>>>>>>>>>>>>>>>>>>>>>>>>> config setting > >>>>>>>>>>>>>>>>>>>>>>>>>>> - For existing ("cold") partitions, the base > >>>>>>>>>>>>>> auto.offset.reset > >>>>>>>>>>>>>>>>>>>>> policy > >>>>>>>>>>>>>>>>>>>>>>>>> continues > >>>>>>>>>>>>>>>>>>>>>>>>>>> to apply unchanged. > >>>>>>>>>>>>>>>>>>>>>>>>>>> - The new-partition reset behavior is > represented > >>>>>> by a > >>>>>>>>>>>>>>> separate > >>>>>>>>>>>>>>>>>>>>>>>>> internal config > >>>>>>>>>>>>>>>>>>>>>>>>>>> (auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> .partitions, > >>>>>>>>>>>>>> currently fixed to > >>>>>>>>>>>>>>>>> earliest). > >>>>>>>>>>>>>>>>>>>>> This > >>>>>>>>>>>>>>>>>>>>>>>>> decoupled design makes > >>>>>>>>>>>>>>>>>>>>>>>>>>> it straightforward to promote the behavior to a > >>>>>> public > >>>>>>>>>>>>>>>>>> user-facing > >>>>>>>>>>>>>>>>>>>>>>>>> configuration in a future KIP. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > 2026年5月16日 > >>>>>> 清晨7:46 > >>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jun > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I see what you mean now. The proposal from me > is > >>>>>> listed > >>>>>>>>>>>>>>> below: > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Add auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> .partitions > >>>>>>>>>>>>>> with a default value > >>>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>>>>>> earliest. It fixes the data loss from both > >>>>>> by_duration and > >>>>>>>>>>>>>>>>> latest, > >>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>>>>>>> does not change the logic of > >>>>>> auto.offset.reset=earliest. > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Mark auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> .partitions > >>>>>>>>>>>>>> as an internal > >>>>>>>>>>>>>>>>>>>>>>>>> configuration. auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> > >>>>>>>>>>>>>> .partitions=earliest > >>>>>>>>>>>>>>> already > >>>>>>>>>>>>>>>>>>>>>>>>> addresses the issue, and we can discuss the use > cases > >>>>>> of > >>>>>>>>>>>>>> other > >>>>>>>>>>>>>>>>>> values > >>>>>>>>>>>>>>>>>>>>>>> in a > >>>>>>>>>>>>>>>>>>>>>>>>> separate KIP. > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) Both configs, auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> .partitions > >>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age.ms > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.latest.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfu9JSP4l$ > >>>>>>> , > >>>>>>>>>>>>>> will be applied to all for > >>>>>>>>>>>>>>>>>>>>>>>>> consistency. > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> WDYT? > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/15 20:53:20 Jun Rao via dev wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, Chia-Ping, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the reply. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. In the motivation section, the KIP says > "When > >>>>>> a Kafka > >>>>>>>>>>>>>>>>> topic > >>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>> expanded > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> with new partitions, consumers using the > latest > >>>>>> auto > >>>>>>>>>>>>>> offset > >>>>>>>>>>>>>>>>>> reset > >>>>>>>>>>>>>>>>>>>>>>>>> policy > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> will silently miss all records produced to > those > >>>>>>>>>>>>>> partitions > >>>>>>>>>>>>>>>>>> before > >>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer discovers them.". If a user sets > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset=by_duration=1sec, the same > >>>>>> record loss > >>>>>>>>>>>>>>>>> issue > >>>>>>>>>>>>>>>>>>>>> could > >>>>>>>>>>>>>>>>>>>>>>>>> also > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> happen, right? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. I was thinking auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> .partitions > >>>>>>>>>>>>>> will > >>>>>>>>>>>>>>> take > >>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>> same > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> values as auto.offset.reset. So a user could > set > >>>>>> it > >>>>>>>>>>>>>>>>>> by_duration if > >>>>>>>>>>>>>>>>>>>>>>>>> needed. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, May 14, 2026 at 4:06 PM Chia-Ping > Tsai < > >>>>>>>>>>>>>>>>>>>>> [email protected] > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jun > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. I might be missing > >>>>>> something > >>>>>>>>>>>>>>>>>> important > >>>>>>>>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>>>>>> your > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion, so please bear with me as I try > to > >>>>>> clarify > >>>>>>>>>>>>>> with > >>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>> few > >>>>>>>>>>>>>>>>>>>>>>>>> questions: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. Is there a strong use case for extending > this > >>>>>> logic > >>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>> other > >>>>>>>>>>>>>>>>>>>>>>> reset > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies? Unlike latest, policies like > earliest > >>>>>> or > >>>>>>>>>>>>>>>>> by_duration > >>>>>>>>>>>>>>>>>>>>>>> don't > >>>>>>>>>>>>>>>>>>>>>>>>> seem > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to suffer from the same silent data loss > issue > >>>>>> when a > >>>>>>>>>>>>>>>>>> partition > >>>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>> expanded. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. What values would we expect users to > >>>>>> configure for > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> .partitions? > >>>>>>>>>>>>>> If they set it to > >>>>>>>>>>>>>>>>> earliest > >>>>>>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>>>>>>> latest, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we might run into the exact same edge > cases. For > >>>>>>>>>>>>>> example, > >>>>>>>>>>>>>>>>> if a > >>>>>>>>>>>>>>>>>>>>>>>>> consumer is > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offline for a while and a new partition is > >>>>>> created > >>>>>>>>>>>>>> during > >>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>> downtime, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the user might actually want to skip to > latest > >>>>>> when > >>>>>>>>>>>>>>>>> resuming, > >>>>>>>>>>>>>>>>>>>>>>> rather > >>>>>>>>>>>>>>>>>>>>>>>>> than > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reading from earliest just because the > partition > >>>>>> is > >>>>>>>>>>>>>>>>>> technically > >>>>>>>>>>>>>>>>>>>>>>>>> "new" to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the group. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is exactly why we opted for > introducing a > >>>>>> max.age > >>>>>>>>>>>>>>>>>> threshold. > >>>>>>>>>>>>>>>>>>>>>>> It > >>>>>>>>>>>>>>>>>>>>>>>>> gives > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users a time-bound way to define what is > >>>>>> genuinely > >>>>>>>>>>>>>>> "hot/new" > >>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>> what is > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> just an old partition they haven't seen yet. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/14 20:48:09 Jun Rao via dev > wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I find auto.offset.reset.latest.max.age a > bit > >>>>>> weird. It > >>>>>>>>>>>>>>>>> only > >>>>>>>>>>>>>>>>>>>>>>>>> applies when > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset is latest. However, it > seems > >>>>>> that the > >>>>>>>>>>>>>>>>>>>>> motivation > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> equally > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applies when auto.offset.reset is set to > other > >>>>>> values > >>>>>>>>>>>>>> like > >>>>>>>>>>>>>>>>>>>>>>>>> by_duration. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intention is that we want to have a > separate > >>>>>> way to > >>>>>>>>>>>>>>> control > >>>>>>>>>>>>>>>>>>>>> newly > >>>>>>>>>>>>>>>>>>>>>>>>> created > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partitions vs existing partitions when the > >>>> group > >>>>>>>>>>>>>> starts. > >>>>>>>>>>>>>>>>>> Have we > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> considered > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding a new config like > auto.offset.reset.new > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > >>>>>>> > >>>>>>>>>>>>>>> .partitions? > >>>>>>>>>>>>>>>>>> If > >>>>>>>>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>>>>>> new > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> config is not set, the offset reset policy > >>>>>> defaults to > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> policy > >>>>>>>>>>>>>>>>>>>>>>>>> used > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing partitions. The user could set it > >>>>>> explicitly > >>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>> customize > >>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior for new partitions. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, May 7, 2026 at 5:07 AM 黃竣陽 < > >>>>>> [email protected] > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I’d like to manually bump this thread. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月1日 > 晚上10:37 > >>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello all, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ01/DJ02: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MetadataResponse bumps from v13 to v14. > The > >>>>>>>>>>>>>>>>>> PartitionMetadata > >>>>>>>>>>>>>>>>>>>>>>>>> struct > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> gains a new > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> field PartitionAgeMs (int64, default -1), > >>>>>> computed > >>>>>>>>>>>>>>>>>> server-side > >>>>>>>>>>>>>>>>>>>>>>> by > >>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> broker as > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> broker_current_time - > >>>> partition_creation_time. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also add the consumer heartbeat flow. > when > >>>>>>>>>>>>>>>>>> MembershipManager > >>>>>>>>>>>>>>>>>>>>>>>>> detects > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly assigned > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition, it explicitly invalidates the > >>>>>> metadata for > >>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>> affected > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> topic > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and forces a fresh MetadataRequest > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before making the offset reset decision, > even > >>>>>> if the > >>>>>>>>>>>>>>>>> topic > >>>>>>>>>>>>>>>>>> ID > >>>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the cache. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MB0: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The consumer learns the broker's maximum > >>>>>> supported > >>>>>>>>>>>>>>>>>>>>>>>>> MetadataResponse > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version via the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ApiVersions negotiation at connection > time. > >>>>>> If the > >>>>>>>>>>>>>>>>>> negotiated > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version is > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unsupported, the consumer > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> knows the broker does not support > >>>>>> PartitionAgeMs at > >>>>>>>>>>>>>> all > >>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>> can > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throw an > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UnsupportedVersionException > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> immediately, rather than silently falling > >>>>>> back to > >>>>>>>>>>>>>> latest > >>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>> risking > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data loss without any operator-visible > signal. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MB1/MB2/MB3: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have addressed these changes in the > KIP. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > >>>>>> 2026年4月29日 > >>>>>>>>>>>>>>> 下午4:04 > >>>>>>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi David > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree with the direction of moving the > >>>>>> 'age' > >>>>>>>>>>>>>>>>> resolution > >>>>>>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Heartbeat API to the Metadata API to keep > the > >>>>>> control > >>>>>>>>>>>>>>>>> plane > >>>>>>>>>>>>>>>>>>>>>>> clean. > >>>>>>>>>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trade-off, as we noted before, is > introducing > >>>>>>>>>>>>>>> inter-broker > >>>>>>>>>>>>>>>>>>>>> clock > >>>>>>>>>>>>>>>>>>>>>>>>> skew. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Group Coordinator approach provided a > single > >>>>>> source of > >>>>>>>>>>>>>>>>> truth > >>>>>>>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>>>>>> time. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However, realistically, this time skew > >>>>>> should be > >>>>>>>>>>>>>>>>>> negligible. > >>>>>>>>>>>>>>>>>>>>>>>>> Given > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the max.age threshold will likely be > >>>>>> configured in > >>>>>>>>>>>>>>> minutes > >>>>>>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>>>>>>> hours, a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> typical NTP skew (in milliseconds) between > >>>>>> brokers > >>>>>>>>>>>>>> won't > >>>>>>>>>>>>>>>>>> impact > >>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fallback decision. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David Jacot via dev < > [email protected]> > >>>>>> 於 > >>>>>>>>>>>>>>>>> 2026年4月29日 > >>>>>>>>>>>>>>>>>>>>>>> 下午3:29 > >>>>>>>>>>>>>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP! > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry, I haven't really followed the > >>>>>> previous > >>>>>>>>>>>>>>>>>> conversation > >>>>>>>>>>>>>>>>>>>>>>> but I > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> took a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quick look at this one. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ01: I don't clearly understand the > flow > >>>>>> with the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ConsumerGroupHeartbeat > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> API after reading the KIP. There is a > new > >>>>>> boolean; > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>> KIP > >>>>>>>>>>>>>>>>>>>>>>>>> states > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition ages are returned only when > this > >>>>>> boolean > >>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>> set. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implicitly, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> means that when the consumer receives > a new > >>>>>>>>>>>>>> partition, > >>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>>> will > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HB request with the boolean set to > receive > >>>>>> the > >>>>>>>>>>>>>> ages. > >>>>>>>>>>>>>>> Is > >>>>>>>>>>>>>>>>>> my > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understanding > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct? We should perhaps clarify the > flow > >>>>>> and > >>>>>>>>>>>>>> also > >>>>>>>>>>>>>>>>>> explain > >>>>>>>>>>>>>>>>>>>>>>>>> how it > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fits > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into the existing flow (e.g. list > offsets, > >>>>>> fetch > >>>>>>>>>>>>>>>>> offsets, > >>>>>>>>>>>>>>>>>>>>>>> etc.). > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ02: It my understanding is correct, I > >>>>>> wonder if > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API is the > right > >>>>>> place > >>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>>>> given > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a new > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> round trip is done anyway. > Alternatively, > >>>>>> it could > >>>>>>>>>>>>>>>>> simply > >>>>>>>>>>>>>>>>>>>>>>>>> include > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metadata. Generally, we should be > rather > >>>>>> cautious > >>>>>>>>>>>>>>> about > >>>>>>>>>>>>>>>>>> not > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overloading > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API with > >>>>>> unrelated > >>>>>>>>>>>>>>> concepts. > >>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>> API > >>>>>>>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control plane API for assigning or > revoking > >>>>>>>>>>>>>>> partitions. > >>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>> fact > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't want to add it to the > corresponding > >>>>>> Streams > >>>>>>>>>>>>>> API > >>>>>>>>>>>>>>>>>> also > >>>>>>>>>>>>>>>>>>>>>>>>> suggests > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> something is not quite right. What > would we > >>>>>> do if > >>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>> want to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Streams in the future? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Apr 29, 2026 at 12:28 AM > >>>>>> Muralidhar Basani > >>>>>>>>>>>>>>> via > >>>>>>>>>>>>>>>>>> dev > >>>>>>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jiunn, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for this great kip. Good to > know > >>>>>> about > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>> gap. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-0 - why a new v2 version bump for > >>>>>>>>>>>>>>>>>> RequestPartitionAges > >>>>>>>>>>>>>>>>>>>>>>>>> field. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tagged field (for ex: on response, > >>>>>> PartitionAges > >>>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TopicPartitions) > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used here and avoid version bump? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-1 - For the new config, is there a > >>>>>> recommended > >>>>>>>>>>>>>>>>> value > >>>>>>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ConfigDef > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> validator? Probably it should based > on the > >>>>>>>>>>>>>>>>>>>>>>> metadata.max.age.ms > >>>>>>>>>>>>>> < > >>>>>> > >>>> > https://urldefense.com/v3/__http://metadata.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkflKEb5SK$ > >>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> ? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sizing > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instructions can be part of javadocs I > >>>>>> guess. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-2 - (minor) As there are no > changes to > >>>>>> Kafka > >>>>>>>>>>>>>>>>> Streams, > >>>>>>>>>>>>>>>>>>>>>>> would > >>>>>>>>>>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to add this new config > >>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age > >>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StreamsConfig block list > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (NON_CONFIGURABLE_CONSUMER_DEFAULT_CONFIGS) > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clear warning, incase users configure > it? > >>>>>> This is > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>> most > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> familiar > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer config and users might easily > >>>>>> mistakenly > >>>>>>>>>>>>>>>>>> configure > >>>>>>>>>>>>>>>>>>>>>>>>> it. Or > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may be > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's not worth it to add. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-3 - (minor) The phrasing "the > consumer > >>>>>> falls > >>>>>>>>>>>>>> back > >>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>> earliest" > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reads as > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if the config were being changed > >>>>>> per-partition > >>>>>>>>>>>>>> which > >>>>>>>>>>>>>>>>>> isn't > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supported. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> May > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be rephrasing to something like > "consumer > >>>>>> resolves > >>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>> initial > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> position to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> start offset for that partition" as if > >>>>>> earliest > >>>>>>>>>>>>>> was > >>>>>>>>>>>>>>>>>> applied > >>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition only and auto.offset.reset > >>>>>> config is > >>>>>>>>>>>>>>>>>> unchanged. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Murali > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Apr 28, 2026 at 2:48 PM 黃竣陽 < > >>>>>>>>>>>>>>>>>> [email protected]> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi chia, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have updated the KIP to include > this > >>>>>> change. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> > 於 > >>>>>>>>>>>>>> 2026年4月28日 > >>>>>>>>>>>>>>>>>> 晚上8:03 > >>>>>>>>>>>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chia_0: Should we expose the > partition > >>>>>> creation > >>>>>>>>>>>>>>> time > >>>>>>>>>>>>>>>>>> via > >>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Admin > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> API? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I assume it would be valuable for > users > >>>> to > >>>>>>>>>>>>>> diagnose > >>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> troubleshoot > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior of > >>>>>> auto.offset.reset.latest.max.age > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/04/28 10:47:58 黃竣陽 wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to start a discussion > on > >>>>>> KIP-1327 > >>>>>>>>>>>>>>>>>> Prevent > >>>>>>>>>>>>>>>>>>>>> Hot > >>>>>>>>>>>>>>>>>>>>>>>>> Data > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Loss > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Partition Expansion for Latest Policy > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>> > >>>> > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/KY4mGQ__;!!Ayb5sqE7!qF4q1QzF1RRgP61D7A2xuEai1ky7fepKDKFFvpNBuePikH-ULmT87TvuuZzy5kau5E4y5zMZAmfQQiwZomM$ > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This proposal aims to introduces > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer config that lets the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest reset policy distinguish > newly > >>>>>> expanded > >>>>>>>>>>>>>>>>> (hot) > >>>>>>>>>>>>>>>>>>>>>>>>> partitions > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> long-existing (cold) ones. Partitions > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> younger than the configured > threshold > >>>>>>>>>>>>>>> automatically > >>>>>>>>>>>>>>>>>> fall > >>>>>>>>>>>>>>>>>>>>>>>>> back > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> earliest, preventing silent data loss > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> during topic expansion without > forcing > >>>>>> a full > >>>>>>>>>>>>>>>>>> historical > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reprocess. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>> > >>>> > >>> > >> > >
