Hi Jiunn-Yang et al., Some background on the problem I hinted at for the KIP-932 implementation. KIP-932 states "If the number of partitions is increased for a topic with a subscription in a share group, the SPSO for the newly created share-partitions is initialized to 0 (which is of course both the earliest and latest offset for an empty topic-partition). This means there is no doubt about what happens when the number of partitions is increased."
The problem was that we had not implemented the use of 0, but instead were using the share.auto.offset.reset config to choose the initial offset in this case. We write a ShareGroupStatePartitionMetadata record when a topic is first being initialized for consumption by a share group, and this contains a list of the partitions which we are initializing. As a result, I believe we can already tell when partitions are added, and we do know when to initialize to 0 as it says in the KIP. We will put in a patch that uses 0 as intended. I do not mind whether we want to incorporate share groups into KIP-1327, but I do not think that it is necessary to do so. Thanks, Andrew On 2026/06/26 04:31:28 Chia-Ping Tsai wrote: > > JR5. Since this changes the on-disk data format for PartitionRecord, we > > need to gate this by a new MV. It also changes the format of > > ConsumerGroupMetadataValue, so we need to gate it too. I'm not sure if this > > requires a new MV or a new GV. > > That is a good point. If we want to maintain consistency for the > NewPartitions field when the group coordinator migrates between new and old > binaries, a version gate is definitely required. For example: > > GV_2(2, MetadataVersion.IBP_4_4_IV0, Map.of()) > > The group coordinator should NOT set the NewPartitions field when the gv is > smaller than gv_2 > > Best, > Chia-Ping > > On 2026/06/25 20:01:39 Jun Rao via dev wrote: > > Hi, Jiunn-Yang, > > > > JR2. We should try to maintain consistency across our APIs. So, it would be > > useful to understand if the user-facing part of this KIP is what share > > consumer wants too. It would be problematic if we proceed with this > > approach only for the share consumer to choose a different one later. > > > > JR3. I was thinking that if we add the NewPartitions field in > > GroupHeartbeatResponse, there is no need to expose the group/partition > > creation timestamp to other RPCs and the Cli. > > > > JR5. Since this changes the on-disk data format for PartitionRecord, we > > need to gate this by a new MV. It also changes the format of > > ConsumerGroupMetadataValue, so we need to gate it too. I'm not sure if this > > requires a new MV or a new GV. > > > > Thanks, > > > > Jun > > > > On Thu, Jun 25, 2026 at 5:52 AM 黃竣陽 <[email protected]> wrote: > > > > > Hello Jun, chia, > > > > > > Thanks for your feedback. > > > > > > JR3 JR4 I have updated the KIP. > > > > > > JR2 The new-partition detection mechanism is reusable. The coordinator > > > already has > > > access to both timestamps required for classification, so the > > > determination can be made > > > server-side without introducing any new RPCs. The main difference is where > > > the policy is > > > enforced: modern consumer resolves it on the client, whereas a share group > > > resolves the > > > starting offset on the partition leader. As a result, the classification > > > decision would need to > > > be propagated through the existing share-state initialization path. We > > > would also need to persist > > > the group's creation time as part of the share-group state, since that > > > information is not stored today. > > > > > > None of this is a blocker, but it does require changes to the share-state > > > format and introduces > > > a share-specific configuration. Given that additional scope, I would > > > prefer to keep share groups out > > > of scope for this KIP and address them in a follow-up effort, while > > > documenting the relevant reuse > > > opportunities here. > > > > > > Best Regards, > > > Jiunn-Yang > > > > > > > Chia-Ping Tsai <[email protected]> 於 2026年6月25日 下午1:31 寫道： > > > > > > > >> It inspires me to think about mirror clusters. What if the mirror group > > > is created 'after' the expanded partitions? In this scenario, the new > > > configuration 'max.age.ms' could handle it if users are aware of the > > > migration and set a suitable time interval. Of course, this would require > > > both creation times to be exposed to clients. > > > > > > > > Thinking more about the mirror cluster scenario we discussed, I realize > > > it introduces significant complexity. Handling it properly would require a > > > tolerance time window rather than a strict time interval to account for > > > replication lag. To prevent scope creep and keep KIP-1327 focused on its > > > primary goal, I think it would be best to defer the > > > multi-cluster/mirroring > > > scenario and explore those approaches in a separate KIP > > > > > > > > On 2026/06/24 21:29:19 Chia-Ping Tsai wrote: > > > >> hi all, > > > >> > > > >>> JR3. The KIP exposes the group/partition creation time to the client. > > > I am > > > >>> not sure if there are other use cases for those timestamps. An > > > alternative > > > >>> is to avoid exposing those timestamps to the client. The group > > > coordinator > > > >>> will categorize the partitions using those timestamps and include the > > > >>> categorization in the assigned partitions in GroupHeartbeatResponse. > > > >> > > > >> It inspires me to think about mirror clusters. What if the mirror group > > > is created 'after' the expanded partitions? In this scenario, the new > > > configuration 'max.age.ms' could handle it if users are aware of the > > > migration and set a suitable time interval. Of course, this would require > > > both creation times to be exposed to clients. > > > >> > > > >> Best, > > > >> Chia-Ping > > > >> > > > >> On 2026/06/24 20:46:16 Jun Rao via dev wrote: > > > >>> Hi, Jiunn-Yang, > > > >>> > > > >>> Thanks for the updated KIP. > > > >>> > > > >>> JR2. Share consumer (KIP-932) states: "If the number of partitions is > > > >>> increased for a topic with a subscription in a share group, the SPSO > > > for > > > >>> the newly created share-partitions is initialized to 0 ". However, > > > >>> this > > > >>> part has not been implemented yet. Currently, all partitions without > > > offset > > > >>> follows the strategy given by share.auto.offset.reset. So, we have an > > > >>> opportunity to see if it's better to align this KIP's changes with the > > > >>> share consumer. > > > >>> JR2.1 The first part is whether customizing the offset for new > > > partitions > > > >>> to something other than 0 is useful in the share consumer. It seems > > > that > > > >>> the motivation in this KIP applies to share consumer too. > > > >>> JR2.2 The second part is how new partitions are detected. The > > > >>> mechanism > > > >>> introduced in this KIP seems quite effective. A group coordinator > > > >>> could > > > >>> pick up newly created partitions for an existing topic or newly > > > >>> created > > > >>> topics matching the regex. Those partitions are categorized as new > > > >>> partitions and will follow the strategy given by > > > >>> auto.offset.reset.new.partitions. > > > >>> This makes sense since those partitions typically don't have a large > > > >>> backlog. A consumer could also additionally subscribe to an existing > > > topic > > > >>> in an existing group. Those partitions will be categorized as existing > > > >>> partitions. This also makes sense since those partitions could have a > > > large > > > >>> backlog and it is better to follow the strategy given by > > > auto.offset.reset. It > > > >>> will be useful to think through whether the logic for categorizing new > > > >>> partitions should also be reused in the share consumer. > > > >>> > > > >>> JR3. The KIP exposes the group/partition creation time to the client. > > > I am > > > >>> not sure if there are other use cases for those timestamps. An > > > alternative > > > >>> is to avoid exposing those timestamps to the client. The group > > > coordinator > > > >>> will categorize the partitions using those timestamps and include the > > > >>> categorization in the assigned partitions in GroupHeartbeatResponse. > > > >>> > > > >>> JR4. It would be useful to support this feature on existing groups as > > > well. > > > >>> For example, any partition without a creation timestamp can always be > > > >>> categorized as an existing partition. > > > >>> > > > >>> Jun > > > >>> > > > >>> On Wed, Jun 24, 2026 at 6:12 AM 黃竣陽 <[email protected]> wrote: > > > >>> > > > >>>> Hello Andrew, > > > >>>> > > > >>>> Thanks for your feedback, > > > >>>>>> I've just been re-reading this KIP to ensure that share group > > > support is > > > >>>>>> not needed. I've discovered that we made a mistake implementing > > > KIP-932 > > > >>>> in > > > >>>>>> this area, so we'll rectify that and check whether it's really > > > >>>>>> tight > > > >>>> enough > > > >>>>>> not to require KIP-1327. > > > >>>> > > > >>>> Could you walk us through the specific area where the mistake was > > > found? > > > >>>> That would > > > >>>> help us assess whether it naturally fits within the scope of KIP-1327 > > > or > > > >>>> should be addressed > > > >>>> separately. > > > >>>> > > > >>>> AS1 I have updated the KIP > > > >>>> > > > >>>> Best Regards, > > > >>>> Jiunn-Yang > > > >>>> > > > >>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月24日 晚上8:49 寫道： > > > >>>>> > > > >>>>> hi Andrew > > > >>>>> > > > >>>>>> I've discovered that we made a mistake implementing KIP-932 in this > > > >>>> area, > > > >>>>> so we'll rectify that and check whether it's really tight enough not > > > to > > > >>>>> require KIP-1327. > > > >>>>> > > > >>>>> Would you mind sharing more details about that? If it's not anything > > > >>>> close > > > >>>>> to NP-hard, we could definitely look into covering it within > > > >>>>> KIP-1327 > > > >>>>> > > > >>>>> Best, > > > >>>>> Chia-Ping > > > >>>>> > > > >>>>> > > > >>>>> Andrew Schofield <[email protected]> 於 2026年6月24日週三 下午7:49寫道： > > > >>>>> > > > >>>>>> Hi Jiunn-Yang, > > > >>>>>> I've just been re-reading this KIP to ensure that share group > > > support is > > > >>>>>> not needed. I've discovered that we made a mistake implementing > > > KIP-932 > > > >>>> in > > > >>>>>> this area, so we'll rectify that and check whether it's really > > > >>>>>> tight > > > >>>> enough > > > >>>>>> not to require KIP-1327. > > > >>>>>> > > > >>>>>> One trivial comments for consistency. > > > >>>>>> > > > >>>>>> AS1: In kafka-consumer-groups.sh, unknown column data is > > > represented by > > > >>>>>> "-", not "N/A". In kafka-topics.sh, we do use "N/A". > > > >>>>>> > > > >>>>>> Thanks, > > > >>>>>> Andrew > > > >>>>>> > > > >>>>>> On 2026/06/23 23:23:38 黃竣陽 wrote: > > > >>>>>>> Hi all, > > > >>>>>>> > > > >>>>>>> Manually bumping this thread. > > > >>>>>>> > > > >>>>>>> Best Regards, > > > >>>>>>> Jiunn-Yang > > > >>>>>>> > > > >>>>>>>> 黃竣陽 <[email protected]> 於 2026年6月17日 晚上9:17 寫道： > > > >>>>>>>> > > > >>>>>>>> Hello chia, > > > >>>>>>>> > > > >>>>>>>> Thanks for the feedback, I have updated the KIP. > > > >>>>>>>> > > > >>>>>>>> Best Regards, > > > >>>>>>>> Jiunn-Yang > > > >>>>>>>> > > > >>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月17日 凌晨12:47 寫道： > > > >>>>>>>>> > > > >>>>>>>>> hi Jiunn-Yang > > > >>>>>>>>> > > > >>>>>>>>>> When the config is set on a cluster that has not yet been > > > >>>>>> upgraded... classification cannot occur... the consumer falls back > > > to > > > >>>> the > > > >>>>>> base auto.offset.reset for the affected partitions. No exception is > > > >>>> thrown, > > > >>>>>> and no operational disruption results. > > > >>>>>>>>> > > > >>>>>>>>> Existing group can't take advantage of this excellent new > > > >>>>>> configuration. Allowing users to modify the group creation time > > > might be > > > >>>>>> overkill. Instead, we could print a useful warning message to guide > > > >>>> users. > > > >>>>>> For example, we can suggest that they re-create the group with > > > >>>>>> their > > > >>>>>> existing committed offsets > > > >>>>>>>>> > > > >>>>>>>>>> Protocol changes > > > >>>>>>>>> > > > >>>>>>>>> Would you mind listing those RPC changes in a table format? > > > >>>>>>>>> > > > >>>>>>>>>> The full interaction matrix between the base policy and the > > > >>>>>> new-partition policy is: > > > >>>>>>>>> > > > >>>>>>>>> Please add a filed to describe the target scenario when using > > > these > > > >>>>>> policies > > > >>>>>>>>> > > > >>>>>>>>> Best, > > > >>>>>>>>> Chia-Ping > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> On 2026/06/16 16:14:49 黃竣陽 wrote: > > > >>>>>>>>>> Hello Jun, chia, > > > >>>>>>>>>> > > > >>>>>>>>>> Thanks for the feedback, I have updated the KIP for the new > > > >>>>>>>>>> approach, PTAL > > > >>>>>>>>>> > > > >>>>>>>>>> Best Regards, > > > >>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>> > > > >>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月16日 上午8:23 寫道： > > > >>>>>>>>>>> > > > >>>>>>>>>>> hi Jun > > > >>>>>>>>>>> > > > >>>>>>>>>>> Yes, your approach is great. I think the combination of latest > > > (for > > > >>>>>> existing partitions) and by_duration (for new partitions) can > > > address > > > >>>> 99% > > > >>>>>> of the complaints I have heard regarding this issue. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Also, leveraging the group creation time here opens the door > > > >>>>>>>>>>> to > > > >>>>>> implementing a new policy based on timestamp seek in the future, > > > should > > > >>>> the > > > >>>>>> community want to pursue that. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Thanks for your patience and constructive feedback. We will > > > update > > > >>>>>> the KIP accordingly. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Best, Chia-Ping > > > >>>>>>>>>>> > > > >>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月16日 清晨5:11 > > > 寫道： > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Hi, Chia-Ping, > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Thanks for the reply. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> I agree that it's probably useful to allow a user to > > > configure a > > > >>>>>> different > > > >>>>>>>>>>>> offset policy for existing partitions vs new partitions. > > > However, > > > >>>>>> using > > > >>>>>>>>>>>> group creation time to capture that seems more intuitive. > > > Here is > > > >>>>>> another > > > >>>>>>>>>>>> proposal: remove auto.offset.reset.max.age.ms and categorize > > > new > > > >>>>>> partitions > > > >>>>>>>>>>>> based on group creation time. Introduce > > > >>>>>>>>>>>> a new config auto.offset.reset.new.partitions whose values > > > can be > > > >>>>>> earliest, > > > >>>>>>>>>>>> latest and by_duration, the same as auto.offset.reset. Users > > > can > > > >>>>>> set > > > >>>>>>>>>>>> `auto.offset.reset.new.partitions` to `earliest` if they > > > want to > > > >>>>>> guarantee > > > >>>>>>>>>>>> no data loss on new partitions. They can also use by_duration > > > to > > > >>>>>> set an > > > >>>>>>>>>>>> upper bound on the backlog replayed, which can be different > > > from > > > >>>>>> that of > > > >>>>>>>>>>>> the existing partitions. This will address your concern about > > > too > > > >>>>>> much > > > >>>>>>>>>>>> backlog being replayed when the offsets are lost. What do you > > > >>>>>> think? > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Jun > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> On Mon, Jun 15, 2026 at 10:39 AM Chia-Ping Tsai < > > > >>>>>> [email protected]> wrote: > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> hi Jun > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> The most important part of this story is how users should > > > expect > > > >>>>>> the data > > > >>>>>>>>>>>>> they can see when using the latest or by_duration policy > > > >>>>>>>>>>>>> with > > > >>>>>> expanded > > > >>>>>>>>>>>>> partitions. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Yes, the by_duration policy can minimize data loss, but it > > > >>>>>>>>>>>>> is > > > >>>>>>>>>>>>> non-deterministic, which means users will either read too > > > many > > > >>>>>> historical > > > >>>>>>>>>>>>> records from existing partitions or lose some records from > > > >>>>>> expanded > > > >>>>>>>>>>>>> partitions. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Also, I agree that auto.offset.reset.max.age.ms > > > >>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > > > >>>>>>> > > > >>>>>>>>>>>>> is a bit hard to understand, and that is why I preferred > > > having a > > > >>>>>> whole new > > > >>>>>>>>>>>>> policy based entirely on group creation time (KIP-1282) > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Best, > > > >>>>>>>>>>>>> Chia-Ping > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月16日週二 > > > 上午1:08寫道： > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Hi, Chia-Ping and Jiunn-Yang, > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Thanks for the reply. I am still trying to understand the > > > value > > > >>>>>> of the new > > > >>>>>>>>>>>>>> configs with the KIP. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> The motivation of the KIP is that a user doesn't want to > > > miss > > > >>>>>> the data if > > > >>>>>>>>>>>>>> the backlog is small. The backlog of the existing partition > > > is > > > >>>>>> easy to > > > >>>>>>>>>>>>>> understand because it relates to retention time. The > > > backlog for > > > >>>>>> the new > > > >>>>>>>>>>>>>> partition is a bit subtle to understand since it depends on > > > the > > > >>>>>> metadata > > > >>>>>>>>>>>>>> refresh delay. To set auto.offset.reset.max.age.ms > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > > > >>>>>>> , > > > >>>>>>>>>>>>>> the user needs to > > > >>>>>>>>>>>>>> understand the metadata refresh delay on the consumer side > > > and > > > >>>>>> use it to > > > >>>>>>>>>>>>>> set the config. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Now, let's consider the alternative: setting the same value > > > for > > > >>>>>> the > > > >>>>>>>>>>>>>> existing by_duration policy. The KIP lists three issues > > > >>>>>>>>>>>>>> with > > > >>>> this > > > >>>>>>>>>>>>>> approach. > > > >>>>>>>>>>>>>> 1. It computes the seek target client-side as now() - > > > duration, > > > >>>>>> which > > > >>>>>>>>>>>>>> introduces clock skew across consumers and forces operators > > > to > > > >>>>>> choose > > > >>>>>>>>>>>>>> overly large durations, causing unnecessary reprocessing. > > > >>>>>>>>>>>>>> 2. The target timestamp is recomputed on each retry, so > > > failed > > > >>>>>>>>>>>>>> ListOffsetsRequest retries can shift the target forward and > > > >>>>>> potentially > > > >>>>>>>>>>>>>> miss records produced between attempts. > > > >>>>>>>>>>>>>> 3. It applies uniformly to all partitions without committed > > > >>>>>> offsets, and > > > >>>>>>>>>>>>>> cannot distinguish newly expanded partitions from > > > long-existing > > > >>>>>> partitions > > > >>>>>>>>>>>>>> newly assigned to the group, leading to unnecessary replay. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Issues 1 and 2 are uncommon and can be mitigated by adding > > > a bit > > > >>>>>> buffer to > > > >>>>>>>>>>>>>> the metadata refresh delay. We could also consider > > > improving the > > > >>>>>>>>>>>>>> implementation. For issue 3, the metadata refresh delay is > > > >>>>>> typically low > > > >>>>>>>>>>>>>> (in the order of minutes with the classic consumer and tens > > > of > > > >>>>>> seconds > > > >>>>>>>>>>>>>> with > > > >>>>>>>>>>>>>> the new consumer). If a user is ok with reading that much > > > >>>>>> backlog for new > > > >>>>>>>>>>>>>> partitions, it seems they will be ok doing the same for > > > existing > > > >>>>>>>>>>>>>> partitions. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> So, instead of introducing a new config, could we just > > > reuse the > > > >>>>>> existing > > > >>>>>>>>>>>>>> config with better documentation and/or implementation? > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Jun > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> On Sat, Jun 13, 2026 at 12:19 AM 黃竣陽 <[email protected]> > > > >>>>>> wrote: > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Hello Jun, > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> You're right that group creation time is the more > > > >>>>>>>>>>>>>>> intuitive > > > >>>>>> answer at > > > >>>>>>>>>>>>>>> first glance, > > > >>>>>>>>>>>>>>> the KIP's own motivation talks about partitions that > > > "predate > > > >>>>>> the group" > > > >>>>>>>>>>>>>>> vs partitions > > > >>>>>>>>>>>>>>> "created during group runtime," which directly points to a > > > >>>>>>>>>>>>>> group-lifecycle > > > >>>>>>>>>>>>>>> classifier. > > > >>>>>>>>>>>>>>> I'd like to walk through why we landed on partition age, > > > and > > > >>>> the > > > >>>>>>>>>>>>>>> trade-offs we considered. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> We evaluated three candidate signals: > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> 1. `by_duration:5secs` > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> This covers the metadata blindness window, but has issues > > > the > > > >>>>>> KIP > > > >>>>>>>>>>>>>>> currently documents > > > >>>>>>>>>>>>>>> under "Why not use `by_duration`?": > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> - Client-side `now() - duration` introduces clock skew > > > across > > > >>>>>> consumers. > > > >>>>>>>>>>>>>>> - `ListOffsets` retries shift the target forward, > > > potentially > > > >>>>>> missing > > > >>>>>>>>>>>>>>> records produced between > > > >>>>>>>>>>>>>>> attempts. > > > >>>>>>>>>>>>>>> - It applies uniformly to all partitions without committed > > > >>>>>> offsets, > > > >>>>>>>>>>>>>>> including pre-existing partitions > > > >>>>>>>>>>>>>>> newly assigned to the group, causing unnecessary replay. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> 2. Group creation time as classifier > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> This works cleanly when the consumer is actively running. > > > Our > > > >>>>>> concern > > > >>>>>>>>>>>>>>> is the idle / late-rejoin case: > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> T=0: Group created. > > > >>>>>>>>>>>>>>> T=1..T=100: Consumer idle (down, disconnected, etc.). > > > >>>>>>>>>>>>>>> T=50: Partition added during the idle window. > > > >>>>>>>>>>>>>>> T=100: Consumer resumes. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Under group creation time, the new partition is classified > > > as > > > >>>>>> new > > > >>>>>>>>>>>>>>> (`50 > 0`) and reset to `earliest`, replaying everything > > > from > > > >>>>>> T=50. > > > >>>>>>>>>>>>>>> But during `[T=1, T=100]`, base partitions also > > > >>>>>>>>>>>>>>> accumulated > > > >>>>>> data that > > > >>>>>>>>>>>>>>> the consumer accepts as lost — that is precisely the > > > contract > > > >>>> of > > > >>>>>>>>>>>>>>> `auto.offset.reset=latest`. There is no principled reason > > > to > > > >>>>>> treat > > > >>>>>>>>>>>>>>> the new partition differently; both contain backlog > > > accumulated > > > >>>>>> during > > > >>>>>>>>>>>>>>> the same idle window. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> This aligns with the "backlog is backlog” principle you > > > raised > > > >>>>>> in > > > >>>>>>>>>>>>>>> the KIP-1282 thread: a `latest` user has tolerated some > > > backlog > > > >>>>>> on > > > >>>>>>>>>>>>>>> every other partition during the same idle period; forcing > > > >>>>>> 0-backlog > > > >>>>>>>>>>>>>>> tolerance only on new partitions would be inconsistent > > > >>>>>>>>>>>>>>> with > > > >>>> that > > > >>>>>>>>>>>>>>> tolerance. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> 3. Partition age vs threshold > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Partition age corresponds to the actual silent data loss > > > >>>> window, > > > >>>>>>>>>>>>>>> the gap between partition creation and the consumer’s > > > metadata > > > >>>>>>>>>>>>>>> refresh. Within this window, data loss is genuinely > > > silent: the > > > >>>>>>>>>>>>>>> consumer had no opportunity to know about the partition. > > > >>>>>> Outside this > > > >>>>>>>>>>>>>>> window, missing data reflects either: > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> - (a) the user’s tolerated cost of running with idle > > > consumers, > > > >>>>>> or > > > >>>>>>>>>>>>>>> - (b) an operational issue to surface via monitoring, not > > > via > > > >>>>>> reset > > > >>>>>>>>>>>>>> policy. > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> We did not choose partition age because it is more elegant > > > than > > > >>>>>> group > > > >>>>>>>>>>>>>>> creation time — we chose it because its failure mode > > > (requires > > > >>>> a > > > >>>>>>>>>>>>>>> threshold) is > > > >>>>>>>>>>>>>>> less invasive than the failure mode of group creation time > > > >>>>>> (overrides > > > >>>>>>>>>>>>>>> user-stated > > > >>>>>>>>>>>>>>> `latest` intent during idle periods). > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月13日 上午11:52 > > > 寫道： > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Hi Jun, > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Relying on both creation times will create an > > > >>>>>>>>>>>>>>>> inconsistent > > > >>>>>> scenario. A > > > >>>>>>>>>>>>>>>> consumer that lost all offsets due to a long sleep will > > > seek > > > >>>>>> to the > > > >>>>>>>>>>>>>>>> beginning for the partitions created later than the > > > >>>>>>>>>>>>>>>> group. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> That is why we initially proposed KIP-1282 to fix the > > > >>>>>> inconsistency > > > >>>>>>>>>>>>>>> using a > > > >>>>>>>>>>>>>>>> whole new policy. Since KIP-1282 couldn't reach a > > > consensus, > > > >>>>>> KIP-1327 > > > >>>>>>>>>>>>>>> goes > > > >>>>>>>>>>>>>>>> back to using flexible configurations to prevent users > > > from > > > >>>>>> falling > > > >>>>>>>>>>>>>> into > > > >>>>>>>>>>>>>>>> that pitfall. > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Best, Chia-Ping > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月13日週六 > > > >>>>>> 上午6:49寫道： > > > >>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Thanks for the reply and sorry for the late reply. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> JR1. The design of auto.offset.reset.max.age.ms > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > > > >>>>>>> > > > >>>>>>>>>>>>>> still feels weird to > > > >>>>>>>>>>>>>>> me. > > > >>>>>>>>>>>>>>>>> It > > > >>>>>>>>>>>>>>>>> categorizes partitions as new or existing based on the > > > >>>>>> partition > > > >>>>>>>>>>>>>>> creation > > > >>>>>>>>>>>>>>>>> time. Intuitively, the categorization should be based on > > > the > > > >>>>>> group > > > >>>>>>>>>>>>>>> creation > > > >>>>>>>>>>>>>>>>> time: all partitions existing when the group is created > > > are > > > >>>>>> existing > > > >>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>> all partitions created after the group creation are new > > > >>>>>> partitions. > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> Jun > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> On Tue, Jun 9, 2026 at 8:51 AM 黃竣陽 <[email protected]> > > > >>>>>> wrote: > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> Hi all, > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> Manually bumping this thread. If there is no further > > > >>>>>>>>>>>>>>>>>> discussion, I will close the vote. > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年6月1日 晚上7:16 寫道： > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> Hello Jian, > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> Thanks for your feedback, > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> Agreed, partition expansion is a common operational > > > task, > > > >>>>>> not an > > > >>>>>>>>>>>>>> edge > > > >>>>>>>>>>>>>>>>>>> case. I've updated the Motivation section accordingly. > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> jian fu <[email protected]> 於 2026年6月1日 下午5:49 寫道： > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Hi Jiunn-Yang: > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Thanks for the KIP. I think it would be useful to > > > clarify > > > >>>>>> that > > > >>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>> is a > > > >>>>>>>>>>>>>>>>>>>> common scenario rather than an edge case, which > > > further > > > >>>>>>>>>>>>>> demonstrates > > > >>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>> need for this optimization. For example: > > > >>>>>>>>>>>>>>>>>>>> A partition expansion is a common operational task in > > > >>>>>> Kafka: To > > > >>>>>>>>>>>>>>>>> balance > > > >>>>>>>>>>>>>>>>>>>> resource utilization and cost, topics are typically > > > >>>>>> created with a > > > >>>>>>>>>>>>>>>>>> moderate > > > >>>>>>>>>>>>>>>>>>>> default partition count. However, as traffic grows > > > over > > > >>>>>> time, it > > > >>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>> often > > > >>>>>>>>>>>>>>>>>>>> necessary to increase the number of partitions to > > > >>>>>> accommodate the > > > >>>>>>>>>>>>>>>>> higher > > > >>>>>>>>>>>>>>>>>>>> workload. > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> Regards > > > >>>>>>>>>>>>>>>>>>>> Jian > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 于2026年5月30日周六 22:31写道： > > > >>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> Hello chia, > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> Thanks for the comments, I have updated the KIP! > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年5月30日 > > > 晚上8:29 > > > >>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> Hi Jiunn-Yang, > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> Would you mind removing the terms "hot" and "cold" > > > when > > > >>>>>>>>>>>>>> describing > > > >>>>>>>>>>>>>>>>>>>>>> partitions in the KIP? I understand you are using > > > them > > > >>>> to > > > >>>>>>>>>>>>>> describe > > > >>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>> "freshness" or the users' need for the records, but > > > >>>>>> applying > > > >>>>>>>>>>>>>> these > > > >>>>>>>>>>>>>>>>>> terms > > > >>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>> the partition itself feels a bit unnatural. > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> After all, in this scenario, users don't really > > > >>>>>>>>>>>>>>>>>>>>>> care > > > >>>>>> whether a > > > >>>>>>>>>>>>>>>>>> partition > > > >>>>>>>>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>>> newly expanded or not. Their only expectation is > > > that > > > >>>>>> they won't > > > >>>>>>>>>>>>>>>>>> silently > > > >>>>>>>>>>>>>>>>>>>>>> lose any live records produced to the topic during > > > their > > > >>>>>> active > > > >>>>>>>>>>>>>>>>>>>>> consumption. > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> Best, Chia-Ping > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月30日週六 下午12:30寫道： > > > >>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> Hello Jun, > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback, I have updated the KIP > > > >>>>>> motivation > > > >>>>>>>>>>>>>>> section. > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 > > > 2026年5月30日 > > > >>>>>> 凌晨1:12 > > > >>>>>>>>>>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> Thanks for the reply. I think we need a stronger > > > >>>>>> motivation > > > >>>>>>>>>>>>>> for > > > >>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>> KIP. > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> The KIP says "The core insight is that not all > > > >>>>>> partitions > > > >>>>>>>>>>>>>> without > > > >>>>>>>>>>>>>>>>> a > > > >>>>>>>>>>>>>>>>>>>>>>>> committed offset are the same. A newly expanded > > > >>>>>> partition > > > >>>>>>>>>>>>>> (hot) > > > >>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>>>>> fundamentally different from a partition the > > > consumer > > > >>>>>> has > > > >>>>>>>>>>>>>> never > > > >>>>>>>>>>>>>>>>> seen > > > >>>>>>>>>>>>>>>>>>>>>>>> because it predates the group (cold)." Why is the > > > hot > > > >>>>>>>>>>>>>> partition > > > >>>>>>>>>>>>>>>>>>>>>>>> fundamentally different from the cold? > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> The KIP says "The existing by_duration policy is > > > also > > > >>>>>>>>>>>>>>> insufficient > > > >>>>>>>>>>>>>>>>>>>>>>> because: > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> - The calculated seek time (now() - duration) > > > varies > > > >>>>>> across > > > >>>>>>>>>>>>>> nodes > > > >>>>>>>>>>>>>>>>>> due > > > >>>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>>> clock skew. To be safe, users must set an overly > > > large > > > >>>>>>>>>>>>>> duration, > > > >>>>>>>>>>>>>>>>>>>>>>> causing > > > >>>>>>>>>>>>>>>>>>>>>>>> unnecessary reprocessing. > > > >>>>>>>>>>>>>>>>>>>>>>>> - On network errors, the client recalculates the > > > seek > > > >>>>>> time on > > > >>>>>>>>>>>>>>>>> retry, > > > >>>>>>>>>>>>>>>>>>>>>>>> shifting the target timestamp forward and risking > > > data > > > >>>>>> loss." > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> However, both of these situations are rare. If > > > these > > > >>>>>> issues > > > >>>>>>>>>>>>>>>>> persist, > > > >>>>>>>>>>>>>>>>>>>>> more > > > >>>>>>>>>>>>>>>>>>>>>>>> severe problems likely exist elsewhere. Rare > > > >>>>>> situations don't > > > >>>>>>>>>>>>>>>>> need a > > > >>>>>>>>>>>>>>>>>>>>>>> common > > > >>>>>>>>>>>>>>>>>>>>>>>> solution. If users care about those rare > > > situations, > > > >>>>>> they can > > > >>>>>>>>>>>>>>>>>> implement > > > >>>>>>>>>>>>>>>>>>>>>>>> customized logic using > > > >>>>>>>>>>>>>>>>>>>>> ConsumerRebalanceListener.onPartitionsAssigned(). > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> Jun > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>> On Sun, May 17, 2026 at 6:50 AM 黃竣陽 < > > > >>>>>> [email protected]> > > > >>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Hello chia, > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback, > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> If the creation time exists, the returned value > > > >>>>>> should > > > >>>>>>>>>>>>>> always > > > >>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>>>>>>>>>> greater > > > >>>>>>>>>>>>>>>>>>>>>>>>> than or equal to zero, right? > > > >>>>>>>>>>>>>>>>>>>>>>>>> I have explicitly mentioned this in the KIP. > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive > > > >>>>>> any > > > >>>>>>>>>>>>>>>>>> field > > > >>>>>>>>>>>>>>>>>>>>>>>>> absent UnsupportedVersionException > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> The earliest point at which we can detect the > > > version > > > >>>>>>>>>>>>>> mismatch > > > >>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>> during > > > >>>>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>> first metadata fetch after assignment, which > > > occurs > > > >>>>>> inside > > > >>>>>>>>>>>>>>>>> poll(). > > > >>>>>>>>>>>>>>>>>>>>>>>>> Therefore, the > > > >>>>>>>>>>>>>>>>>>>>>>>>> user would encounter an > > > UnsupportedVersionException > > > >>>>>> from > > > >>>>>>>>>>>>>> poll(). > > > >>>>>>>>>>>>>>>>>> I’ll > > > >>>>>>>>>>>>>>>>>>>>>>>>> clarify this in the KIP. > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > > > 2026年5月17日 > > > >>>>>> 下午4:50 > > > >>>>>>>>>>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> hi Jiunn > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> PartitionAgeMs (int64, default -1): The age of > > > this > > > >>>>>>>>>>>>>> partition > > > >>>>>>>>>>>>>>>>> in > > > >>>>>>>>>>>>>>>>>>>>>>>>> milliseconds, computed server-side by the broker > > > as > > > >>>>>>>>>>>>>>>>>>>>> broker_current_time > > > >>>>>>>>>>>>>>>>>>>>>>> - > > > >>>>>>>>>>>>>>>>>>>>>>>>> partition_creation_time. Returns -1 if the > > > >>>>>>>>>>>>>>>>>>>>>>>>> broker > > > >>>>>> does not > > > >>>>>>>>>>>>>>>>> support > > > >>>>>>>>>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>>>>>>>>>> feature or the partition creation time is > > > unknown. > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> If the creation time exists, the returned value > > > >>>>>> should > > > >>>>>>>>>>>>>> always > > > >>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>>>>>>>>>> greater > > > >>>>>>>>>>>>>>>>>>>>>>>>> than or equal to zero, right? > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive > > > >>>>>> any > > > >>>>>>>>>>>>>>>>>> field > > > >>>>>>>>>>>>>>>>>>>>>>>>> absent UnsupportedVersionException > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Will user encounter UnsupportedVersionException > > > when > > > >>>>>> calling > > > >>>>>>>>>>>>>>>>>>>>> `poll()`? > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Best, > > > >>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/16 04:30:49 黃竣陽 wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Hello Jun, chia, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> I've updated KIP-1327 with a design change > > > based on > > > >>>>>> the > > > >>>>>>>>>>>>>>>>>> discussion > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> feedback. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> The updated design decouples the new-partition > > > >>>> reset > > > >>>>>>>>>>>>>> behavior > > > >>>>>>>>>>>>>>>>>> from > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> the base auto.offset.reset policy: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> - auto.offset.reset.max.age.ms > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$ > > > >>>>>>> > > > >>>>>>>>>>>>>> now applies to all > > > >>>>>>>>>>>>>>>>>> auto.offset.reset > > > >>>>>>>>>>>>>>>>>>>>>>>>> values > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> (latest, earliest, by_duration, none). > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> - For new ("hot") partitions, the consumer > > > resets > > > >>>> to > > > >>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> > > > >>>>>>>>>>>>>> .partitions > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> config setting > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> - For existing ("cold") partitions, the base > > > >>>>>>>>>>>>>> auto.offset.reset > > > >>>>>>>>>>>>>>>>>>>>> policy > > > >>>>>>>>>>>>>>>>>>>>>>>>> continues > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> to apply unchanged. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> - The new-partition reset behavior is > > > represented > > > >>>>>> by a > > > >>>>>>>>>>>>>>> separate > > > >>>>>>>>>>>>>>>>>>>>>>>>> internal config > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> (auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> .partitions, > > > >>>>>>>>>>>>>> currently fixed to > > > >>>>>>>>>>>>>>>>> earliest). > > > >>>>>>>>>>>>>>>>>>>>> This > > > >>>>>>>>>>>>>>>>>>>>>>>>> decoupled design makes > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> it straightforward to promote the behavior to > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> a > > > >>>>>> public > > > >>>>>>>>>>>>>>>>>> user-facing > > > >>>>>>>>>>>>>>>>>>>>>>>>> configuration in a future KIP. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > > > 2026年5月16日 > > > >>>>>> 清晨7:46 > > > >>>>>>>>>>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jun > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I see what you mean now. The proposal from me > > > is > > > >>>>>> listed > > > >>>>>>>>>>>>>>> below: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Add auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> .partitions > > > >>>>>>>>>>>>>> with a default value > > > >>>>>>>>>>>>>>>>> of > > > >>>>>>>>>>>>>>>>>>>>>>>>> earliest. It fixes the data loss from both > > > >>>>>> by_duration and > > > >>>>>>>>>>>>>>>>> latest, > > > >>>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>>>>>> it > > > >>>>>>>>>>>>>>>>>>>>>>>>> does not change the logic of > > > >>>>>> auto.offset.reset=earliest. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Mark auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> .partitions > > > >>>>>>>>>>>>>> as an internal > > > >>>>>>>>>>>>>>>>>>>>>>>>> configuration. auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> > > > >>>>>>>>>>>>>> .partitions=earliest > > > >>>>>>>>>>>>>>> already > > > >>>>>>>>>>>>>>>>>>>>>>>>> addresses the issue, and we can discuss the use > > > cases > > > >>>>>> of > > > >>>>>>>>>>>>>> other > > > >>>>>>>>>>>>>>>>>> values > > > >>>>>>>>>>>>>>>>>>>>>>> in a > > > >>>>>>>>>>>>>>>>>>>>>>>>> separate KIP. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) Both configs, auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> .partitions > > > >>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age.ms > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.latest.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfu9JSP4l$ > > > >>>>>>> , > > > >>>>>>>>>>>>>> will be applied to all for > > > >>>>>>>>>>>>>>>>>>>>>>>>> consistency. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> WDYT? > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/15 20:53:20 Jun Rao via dev wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, Chia-Ping, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the reply. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. In the motivation section, the KIP says > > > "When > > > >>>>>> a Kafka > > > >>>>>>>>>>>>>>>>> topic > > > >>>>>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>>>>>> expanded > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> with new partitions, consumers using the > > > latest > > > >>>>>> auto > > > >>>>>>>>>>>>>> offset > > > >>>>>>>>>>>>>>>>>> reset > > > >>>>>>>>>>>>>>>>>>>>>>>>> policy > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> will silently miss all records produced to > > > those > > > >>>>>>>>>>>>>> partitions > > > >>>>>>>>>>>>>>>>>> before > > > >>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer discovers them.". If a user sets > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset=by_duration=1sec, the same > > > >>>>>> record loss > > > >>>>>>>>>>>>>>>>> issue > > > >>>>>>>>>>>>>>>>>>>>> could > > > >>>>>>>>>>>>>>>>>>>>>>>>> also > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> happen, right? > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. I was thinking auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> .partitions > > > >>>>>>>>>>>>>> will > > > >>>>>>>>>>>>>>> take > > > >>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>> same > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> values as auto.offset.reset. So a user could > > > set > > > >>>>>> it > > > >>>>>>>>>>>>>>>>>> by_duration if > > > >>>>>>>>>>>>>>>>>>>>>>>>> needed. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jun > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, May 14, 2026 at 4:06 PM Chia-Ping > > > Tsai < > > > >>>>>>>>>>>>>>>>>>>>> [email protected] > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jun > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. I might be missing > > > >>>>>> something > > > >>>>>>>>>>>>>>>>>> important > > > >>>>>>>>>>>>>>>>>>>>>>> from > > > >>>>>>>>>>>>>>>>>>>>>>>>> your > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion, so please bear with me as I try > > > to > > > >>>>>> clarify > > > >>>>>>>>>>>>>> with > > > >>>>>>>>>>>>>>>>> a > > > >>>>>>>>>>>>>>>>>> few > > > >>>>>>>>>>>>>>>>>>>>>>>>> questions: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. Is there a strong use case for extending > > > this > > > >>>>>> logic > > > >>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>> other > > > >>>>>>>>>>>>>>>>>>>>>>> reset > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> policies? Unlike latest, policies like > > > earliest > > > >>>>>> or > > > >>>>>>>>>>>>>>>>> by_duration > > > >>>>>>>>>>>>>>>>>>>>>>> don't > > > >>>>>>>>>>>>>>>>>>>>>>>>> seem > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to suffer from the same silent data loss > > > issue > > > >>>>>> when a > > > >>>>>>>>>>>>>>>>>> partition > > > >>>>>>>>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>>>>>> expanded. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. What values would we expect users to > > > >>>>>> configure for > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> .partitions? > > > >>>>>>>>>>>>>> If they set it to > > > >>>>>>>>>>>>>>>>> earliest > > > >>>>>>>>>>>>>>>>>> or > > > >>>>>>>>>>>>>>>>>>>>>>>>> latest, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we might run into the exact same edge > > > cases. For > > > >>>>>>>>>>>>>> example, > > > >>>>>>>>>>>>>>>>> if a > > > >>>>>>>>>>>>>>>>>>>>>>>>> consumer is > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offline for a while and a new partition is > > > >>>>>> created > > > >>>>>>>>>>>>>> during > > > >>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>>>>>>>> downtime, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the user might actually want to skip to > > > latest > > > >>>>>> when > > > >>>>>>>>>>>>>>>>> resuming, > > > >>>>>>>>>>>>>>>>>>>>>>> rather > > > >>>>>>>>>>>>>>>>>>>>>>>>> than > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reading from earliest just because the > > > partition > > > >>>>>> is > > > >>>>>>>>>>>>>>>>>> technically > > > >>>>>>>>>>>>>>>>>>>>>>>>> "new" to > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the group. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is exactly why we opted for > > > introducing a > > > >>>>>> max.age > > > >>>>>>>>>>>>>>>>>> threshold. > > > >>>>>>>>>>>>>>>>>>>>>>> It > > > >>>>>>>>>>>>>>>>>>>>>>>>> gives > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> users a time-bound way to define what is > > > >>>>>> genuinely > > > >>>>>>>>>>>>>>> "hot/new" > > > >>>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>>>>>>>> what is > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> just an old partition they haven't seen > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> yet. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/14 20:48:09 Jun Rao via dev > > > wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I find auto.offset.reset.latest.max.age a > > > bit > > > >>>>>> weird. It > > > >>>>>>>>>>>>>>>>> only > > > >>>>>>>>>>>>>>>>>>>>>>>>> applies when > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset is latest. However, it > > > seems > > > >>>>>> that the > > > >>>>>>>>>>>>>>>>>>>>> motivation > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> equally > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> applies when auto.offset.reset is set to > > > other > > > >>>>>> values > > > >>>>>>>>>>>>>> like > > > >>>>>>>>>>>>>>>>>>>>>>>>> by_duration. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> intention is that we want to have a > > > separate > > > >>>>>> way to > > > >>>>>>>>>>>>>>> control > > > >>>>>>>>>>>>>>>>>>>>> newly > > > >>>>>>>>>>>>>>>>>>>>>>>>> created > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partitions vs existing partitions when the > > > >>>> group > > > >>>>>>>>>>>>>> starts. > > > >>>>>>>>>>>>>>>>>> Have we > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> considered > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding a new config like > > > auto.offset.reset.new > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$ > > > >>>>>>> > > > >>>>>>>>>>>>>>> .partitions? > > > >>>>>>>>>>>>>>>>>> If > > > >>>>>>>>>>>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>>>>>>>>>> new > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> config is not set, the offset reset policy > > > >>>>>> defaults to > > > >>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>> policy > > > >>>>>>>>>>>>>>>>>>>>>>>>> used > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing partitions. The user could set it > > > >>>>>> explicitly > > > >>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>> customize > > > >>>>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior for new partitions. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jun > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, May 7, 2026 at 5:07 AM 黃竣陽 < > > > >>>>>> [email protected] > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I’d like to manually bump this thread. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月1日 > > > 晚上10:37 > > > >>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello all, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ01/DJ02: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MetadataResponse bumps from v13 to v14. > > > The > > > >>>>>>>>>>>>>>>>>> PartitionMetadata > > > >>>>>>>>>>>>>>>>>>>>>>>>> struct > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> gains a new > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> field PartitionAgeMs (int64, default > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -1), > > > >>>>>> computed > > > >>>>>>>>>>>>>>>>>> server-side > > > >>>>>>>>>>>>>>>>>>>>>>> by > > > >>>>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> broker as > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> broker_current_time - > > > >>>> partition_creation_time. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also add the consumer heartbeat flow. > > > when > > > >>>>>>>>>>>>>>>>>> MembershipManager > > > >>>>>>>>>>>>>>>>>>>>>>>>> detects > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> newly assigned > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition, it explicitly invalidates the > > > >>>>>> metadata for > > > >>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>> affected > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> topic > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and forces a fresh MetadataRequest > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before making the offset reset decision, > > > even > > > >>>>>> if the > > > >>>>>>>>>>>>>>>>> topic > > > >>>>>>>>>>>>>>>>>> ID > > > >>>>>>>>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the cache. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MB0: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The consumer learns the broker's maximum > > > >>>>>> supported > > > >>>>>>>>>>>>>>>>>>>>>>>>> MetadataResponse > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version via the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ApiVersions negotiation at connection > > > time. > > > >>>>>> If the > > > >>>>>>>>>>>>>>>>>> negotiated > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> version is > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unsupported, the consumer > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> knows the broker does not support > > > >>>>>> PartitionAgeMs at > > > >>>>>>>>>>>>>> all > > > >>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>>>> can > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> throw an > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UnsupportedVersionException > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> immediately, rather than silently > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> falling > > > >>>>>> back to > > > >>>>>>>>>>>>>> latest > > > >>>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>>>>>>>> risking > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data loss without any operator-visible > > > signal. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MB1/MB2/MB3: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have addressed these changes in the > > > KIP. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > > > >>>>>> 2026年4月29日 > > > >>>>>>>>>>>>>>> 下午4:04 > > > >>>>>>>>>>>>>>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi David > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree with the direction of moving > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>> 'age' > > > >>>>>>>>>>>>>>>>> resolution > > > >>>>>>>>>>>>>>>>>>>>> from > > > >>>>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Heartbeat API to the Metadata API to keep > > > the > > > >>>>>> control > > > >>>>>>>>>>>>>>>>> plane > > > >>>>>>>>>>>>>>>>>>>>>>> clean. > > > >>>>>>>>>>>>>>>>>>>>>>>>> The > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> main > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trade-off, as we noted before, is > > > introducing > > > >>>>>>>>>>>>>>> inter-broker > > > >>>>>>>>>>>>>>>>>>>>> clock > > > >>>>>>>>>>>>>>>>>>>>>>>>> skew. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Group Coordinator approach provided a > > > single > > > >>>>>> source of > > > >>>>>>>>>>>>>>>>> truth > > > >>>>>>>>>>>>>>>>>>>>> for > > > >>>>>>>>>>>>>>>>>>>>>>>>> time. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However, realistically, this time skew > > > >>>>>> should be > > > >>>>>>>>>>>>>>>>>> negligible. > > > >>>>>>>>>>>>>>>>>>>>>>>>> Given > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the max.age threshold will likely be > > > >>>>>> configured in > > > >>>>>>>>>>>>>>> minutes > > > >>>>>>>>>>>>>>>>>> or > > > >>>>>>>>>>>>>>>>>>>>>>>>> hours, a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> typical NTP skew (in milliseconds) > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> between > > > >>>>>> brokers > > > >>>>>>>>>>>>>> won't > > > >>>>>>>>>>>>>>>>>> impact > > > >>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fallback decision. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David Jacot via dev < > > > [email protected]> > > > >>>>>> 於 > > > >>>>>>>>>>>>>>>>> 2026年4月29日 > > > >>>>>>>>>>>>>>>>>>>>>>> 下午3:29 > > > >>>>>>>>>>>>>>>>>>>>>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP! > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry, I haven't really followed the > > > >>>>>> previous > > > >>>>>>>>>>>>>>>>>> conversation > > > >>>>>>>>>>>>>>>>>>>>>>> but I > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> took a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quick look at this one. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ01: I don't clearly understand the > > > flow > > > >>>>>> with the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ConsumerGroupHeartbeat > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> API after reading the KIP. There is a > > > new > > > >>>>>> boolean; > > > >>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>> KIP > > > >>>>>>>>>>>>>>>>>>>>>>>>> states > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition ages are returned only when > > > this > > > >>>>>> boolean > > > >>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>> set. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implicitly, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> means that when the consumer receives > > > a new > > > >>>>>>>>>>>>>> partition, > > > >>>>>>>>>>>>>>>>> it > > > >>>>>>>>>>>>>>>>>>>>> will > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HB request with the boolean set to > > > receive > > > >>>>>> the > > > >>>>>>>>>>>>>> ages. > > > >>>>>>>>>>>>>>> Is > > > >>>>>>>>>>>>>>>>>> my > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> understanding > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct? We should perhaps clarify the > > > flow > > > >>>>>> and > > > >>>>>>>>>>>>>> also > > > >>>>>>>>>>>>>>>>>> explain > > > >>>>>>>>>>>>>>>>>>>>>>>>> how it > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fits > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into the existing flow (e.g. list > > > offsets, > > > >>>>>> fetch > > > >>>>>>>>>>>>>>>>> offsets, > > > >>>>>>>>>>>>>>>>>>>>>>> etc.). > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ02: It my understanding is correct, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I > > > >>>>>> wonder if > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API is the > > > right > > > >>>>>> place > > > >>>>>>>>>>>>>> for > > > >>>>>>>>>>>>>>>>>> this > > > >>>>>>>>>>>>>>>>>>>>>>> given > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a new > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> round trip is done anyway. > > > Alternatively, > > > >>>>>> it could > > > >>>>>>>>>>>>>>>>> simply > > > >>>>>>>>>>>>>>>>>>>>>>>>> include > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metadata. Generally, we should be > > > rather > > > >>>>>> cautious > > > >>>>>>>>>>>>>>> about > > > >>>>>>>>>>>>>>>>>> not > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> overloading > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API with > > > >>>>>> unrelated > > > >>>>>>>>>>>>>>> concepts. > > > >>>>>>>>>>>>>>>>>> The > > > >>>>>>>>>>>>>>>>>>>>>>> API > > > >>>>>>>>>>>>>>>>>>>>>>>>> is > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control plane API for assigning or > > > revoking > > > >>>>>>>>>>>>>>> partitions. > > > >>>>>>>>>>>>>>>>>> The > > > >>>>>>>>>>>>>>>>>>>>>>> fact > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't want to add it to the > > > corresponding > > > >>>>>> Streams > > > >>>>>>>>>>>>>> API > > > >>>>>>>>>>>>>>>>>> also > > > >>>>>>>>>>>>>>>>>>>>>>>>> suggests > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> something is not quite right. What > > > would we > > > >>>>>> do if > > > >>>>>>>>>>>>>> we > > > >>>>>>>>>>>>>>>>>> want to > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> support > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Streams in the future? > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Apr 29, 2026 at 12:28 AM > > > >>>>>> Muralidhar Basani > > > >>>>>>>>>>>>>>> via > > > >>>>>>>>>>>>>>>>>> dev > > > >>>>>>>>>>>>>>>>>>>>> < > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jiunn, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for this great kip. Good to > > > know > > > >>>>>> about > > > >>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>> gap. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-0 - why a new v2 version bump for > > > >>>>>>>>>>>>>>>>>> RequestPartitionAges > > > >>>>>>>>>>>>>>>>>>>>>>>>> field. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tagged field (for ex: on response, > > > >>>>>> PartitionAges > > > >>>>>>>>>>>>>> on > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> TopicPartitions) > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used here and avoid version bump? > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-1 - For the new config, is there a > > > >>>>>> recommended > > > >>>>>>>>>>>>>>>>> value > > > >>>>>>>>>>>>>>>>>> or > > > >>>>>>>>>>>>>>>>>>>>> a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ConfigDef > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> validator? Probably it should based > > > on the > > > >>>>>>>>>>>>>>>>>>>>>>> metadata.max.age.ms > > > >>>>>>>>>>>>>> < > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__http://metadata.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkflKEb5SK$ > > > >>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> ? > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sizing > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instructions can be part of javadocs > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I > > > >>>>>> guess. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-2 - (minor) As there are no > > > changes to > > > >>>>>> Kafka > > > >>>>>>>>>>>>>>>>> Streams, > > > >>>>>>>>>>>>>>>>>>>>>>> would > > > >>>>>>>>>>>>>>>>>>>>>>>>> it > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> better > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to add this new config > > > >>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age > > > >>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StreamsConfig block list > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (NON_CONFIGURABLE_CONSUMER_DEFAULT_CONFIGS) > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clear warning, incase users configure > > > it? > > > >>>>>> This is > > > >>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>> most > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> familiar > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer config and users might > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> easily > > > >>>>>> mistakenly > > > >>>>>>>>>>>>>>>>>> configure > > > >>>>>>>>>>>>>>>>>>>>>>>>> it. Or > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may be > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's not worth it to add. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-3 - (minor) The phrasing "the > > > consumer > > > >>>>>> falls > > > >>>>>>>>>>>>>> back > > > >>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>>>> earliest" > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reads as > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if the config were being changed > > > >>>>>> per-partition > > > >>>>>>>>>>>>>> which > > > >>>>>>>>>>>>>>>>>> isn't > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> supported. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> May > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be rephrasing to something like > > > "consumer > > > >>>>>> resolves > > > >>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>> initial > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> position to > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> start offset for that partition" as > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if > > > >>>>>> earliest > > > >>>>>>>>>>>>>> was > > > >>>>>>>>>>>>>>>>>> applied > > > >>>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition only and auto.offset.reset > > > >>>>>> config is > > > >>>>>>>>>>>>>>>>>> unchanged. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Murali > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Apr 28, 2026 at 2:48 PM 黃竣陽 > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> < > > > >>>>>>>>>>>>>>>>>> [email protected]> > > > >>>>>>>>>>>>>>>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi chia, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have updated the KIP to include > > > this > > > >>>>>> change. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <[email protected]> > > > 於 > > > >>>>>>>>>>>>>> 2026年4月28日 > > > >>>>>>>>>>>>>>>>>> 晚上8:03 > > > >>>>>>>>>>>>>>>>>>>>>>> 寫道： > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chia_0: Should we expose the > > > partition > > > >>>>>> creation > > > >>>>>>>>>>>>>>> time > > > >>>>>>>>>>>>>>>>>> via > > > >>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Admin > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> API? > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I assume it would be valuable for > > > users > > > >>>> to > > > >>>>>>>>>>>>>> diagnose > > > >>>>>>>>>>>>>>>>> and > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> troubleshoot > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior of > > > >>>>>> auto.offset.reset.latest.max.age > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/04/28 10:47:58 黃竣陽 wrote: > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to start a discussion > > > on > > > >>>>>> KIP-1327 > > > >>>>>>>>>>>>>>>>>> Prevent > > > >>>>>>>>>>>>>>>>>>>>> Hot > > > >>>>>>>>>>>>>>>>>>>>>>>>> Data > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Loss > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Partition Expansion for Latest > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Policy > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> < > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>> > > > >>>> > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/KY4mGQ__;!!Ayb5sqE7!qF4q1QzF1RRgP61D7A2xuEai1ky7fepKDKFFvpNBuePikH-ULmT87TvuuZzy5kau5E4y5zMZAmfQQiwZomM$ > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This proposal aims to introduces > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer config that lets the > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest reset policy distinguish > > > newly > > > >>>>>> expanded > > > >>>>>>>>>>>>>>>>> (hot) > > > >>>>>>>>>>>>>>>>>>>>>>>>> partitions > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> long-existing (cold) ones. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Partitions > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> younger than the configured > > > threshold > > > >>>>>>>>>>>>>>> automatically > > > >>>>>>>>>>>>>>>>>> fall > > > >>>>>>>>>>>>>>>>>>>>>>>>> back > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> earliest, preventing silent data > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> loss > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> during topic expansion without > > > forcing > > > >>>>>> a full > > > >>>>>>>>>>>>>>>>>> historical > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reprocess. > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>>> > > > >>> > > > >> > > > > > > > > >
