Hello Jun, chia, Thanks for the feedback, I have updated the KIP for the new approach, PTAL
Best Regards, Jiunn-Yang > Chia-Ping Tsai <[email protected]> 於 2026年6月16日 上午8:23 寫道： > > hi Jun > > Yes, your approach is great. I think the combination of latest (for existing > partitions) and by_duration (for new partitions) can address 99% of the > complaints I have heard regarding this issue. > > Also, leveraging the group creation time here opens the door to implementing > a new policy based on timestamp seek in the future, should the community want > to pursue that. > > Thanks for your patience and constructive feedback. We will update the KIP > accordingly. > > Best, Chia-Ping > >> Jun Rao via dev <[email protected]> 於 2026年6月16日 清晨5:11 寫道： >> >> Hi, Chia-Ping, >> >> Thanks for the reply. >> >> I agree that it's probably useful to allow a user to configure a different >> offset policy for existing partitions vs new partitions. However, using >> group creation time to capture that seems more intuitive. Here is another >> proposal: remove auto.offset.reset.max.age.ms and categorize new partitions >> based on group creation time. Introduce >> a new config auto.offset.reset.new.partitions whose values can be earliest, >> latest and by_duration, the same as auto.offset.reset. Users can set >> `auto.offset.reset.new.partitions` to `earliest` if they want to guarantee >> no data loss on new partitions. They can also use by_duration to set an >> upper bound on the backlog replayed, which can be different from that of >> the existing partitions. This will address your concern about too much >> backlog being replayed when the offsets are lost. What do you think? >> >> Jun >> >> >>> On Mon, Jun 15, 2026 at 10:39 AM Chia-Ping Tsai <[email protected]> wrote: >>> >>> hi Jun >>> >>> The most important part of this story is how users should expect the data >>> they can see when using the latest or by_duration policy with expanded >>> partitions. >>> >>> Yes, the by_duration policy can minimize data loss, but it is >>> non-deterministic, which means users will either read too many historical >>> records from existing partitions or lose some records from expanded >>> partitions. >>> >>> Also, I agree that auto.offset.reset.max.age.ms >>> <https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$> >>> is a bit hard to understand, and that is why I preferred having a whole new >>> policy based entirely on group creation time (KIP-1282) >>> >>> Best, >>> Chia-Ping >>> >>> Jun Rao via dev <[email protected]> 於 2026年6月16日週二 上午1:08寫道： >>> >>>> Hi, Chia-Ping and Jiunn-Yang, >>>> >>>> Thanks for the reply. I am still trying to understand the value of the new >>>> configs with the KIP. >>>> >>>> The motivation of the KIP is that a user doesn't want to miss the data if >>>> the backlog is small. The backlog of the existing partition is easy to >>>> understand because it relates to retention time. The backlog for the new >>>> partition is a bit subtle to understand since it depends on the metadata >>>> refresh delay. To set auto.offset.reset.max.age.ms >>>> <https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$>, >>>> the user needs to >>>> understand the metadata refresh delay on the consumer side and use it to >>>> set the config. >>>> >>>> Now, let's consider the alternative: setting the same value for the >>>> existing by_duration policy. The KIP lists three issues with this >>>> approach. >>>> 1. It computes the seek target client-side as now() - duration, which >>>> introduces clock skew across consumers and forces operators to choose >>>> overly large durations, causing unnecessary reprocessing. >>>> 2. The target timestamp is recomputed on each retry, so failed >>>> ListOffsetsRequest retries can shift the target forward and potentially >>>> miss records produced between attempts. >>>> 3. It applies uniformly to all partitions without committed offsets, and >>>> cannot distinguish newly expanded partitions from long-existing partitions >>>> newly assigned to the group, leading to unnecessary replay. >>>> >>>> Issues 1 and 2 are uncommon and can be mitigated by adding a bit buffer to >>>> the metadata refresh delay. We could also consider improving the >>>> implementation. For issue 3, the metadata refresh delay is typically low >>>> (in the order of minutes with the classic consumer and tens of seconds >>>> with >>>> the new consumer). If a user is ok with reading that much backlog for new >>>> partitions, it seems they will be ok doing the same for existing >>>> partitions. >>>> >>>> So, instead of introducing a new config, could we just reuse the existing >>>> config with better documentation and/or implementation? >>>> >>>> Jun >>>> >>>> >>>>> On Sat, Jun 13, 2026 at 12:19 AM 黃竣陽 <[email protected]> wrote: >>>> >>>>> Hello Jun, >>>>> >>>>> You're right that group creation time is the more intuitive answer at >>>>> first glance, >>>>> the KIP's own motivation talks about partitions that "predate the group" >>>>> vs partitions >>>>> "created during group runtime," which directly points to a >>>> group-lifecycle >>>>> classifier. >>>>> I'd like to walk through why we landed on partition age, and the >>>>> trade-offs we considered. >>>>> >>>>> We evaluated three candidate signals: >>>>> >>>>> 1. `by_duration:5secs` >>>>> >>>>> This covers the metadata blindness window, but has issues the KIP >>>>> currently documents >>>>> under "Why not use `by_duration`?": >>>>> >>>>> - Client-side `now() - duration` introduces clock skew across consumers. >>>>> - `ListOffsets` retries shift the target forward, potentially missing >>>>> records produced between >>>>> attempts. >>>>> - It applies uniformly to all partitions without committed offsets, >>>>> including pre-existing partitions >>>>> newly assigned to the group, causing unnecessary replay. >>>>> >>>>> 2. Group creation time as classifier >>>>> >>>>> This works cleanly when the consumer is actively running. Our concern >>>>> is the idle / late-rejoin case: >>>>> >>>>> T=0: Group created. >>>>> T=1..T=100: Consumer idle (down, disconnected, etc.). >>>>> T=50: Partition added during the idle window. >>>>> T=100: Consumer resumes. >>>>> >>>>> Under group creation time, the new partition is classified as new >>>>> (`50 > 0`) and reset to `earliest`, replaying everything from T=50. >>>>> But during `[T=1, T=100]`, base partitions also accumulated data that >>>>> the consumer accepts as lost — that is precisely the contract of >>>>> `auto.offset.reset=latest`. There is no principled reason to treat >>>>> the new partition differently; both contain backlog accumulated during >>>>> the same idle window. >>>>> >>>>> This aligns with the "backlog is backlog” principle you raised in >>>>> the KIP-1282 thread: a `latest` user has tolerated some backlog on >>>>> every other partition during the same idle period; forcing 0-backlog >>>>> tolerance only on new partitions would be inconsistent with that >>>>> tolerance. >>>>> >>>>> 3. Partition age vs threshold >>>>> >>>>> Partition age corresponds to the actual silent data loss window, >>>>> the gap between partition creation and the consumer’s metadata >>>>> refresh. Within this window, data loss is genuinely silent: the >>>>> consumer had no opportunity to know about the partition. Outside this >>>>> window, missing data reflects either: >>>>> >>>>> - (a) the user’s tolerated cost of running with idle consumers, or >>>>> - (b) an operational issue to surface via monitoring, not via reset >>>> policy. >>>>> >>>>> We did not choose partition age because it is more elegant than group >>>>> creation time — we chose it because its failure mode (requires a >>>>> threshold) is >>>>> less invasive than the failure mode of group creation time (overrides >>>>> user-stated >>>>> `latest` intent during idle periods). >>>>> >>>>> Best Regards, >>>>> Jiunn-Yang >>>>> >>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月13日 上午11:52 寫道： >>>>>> >>>>>> Hi Jun, >>>>>> >>>>>> Relying on both creation times will create an inconsistent scenario. A >>>>>> consumer that lost all offsets due to a long sleep will seek to the >>>>>> beginning for the partitions created later than the group. >>>>>> >>>>>> That is why we initially proposed KIP-1282 to fix the inconsistency >>>>> using a >>>>>> whole new policy. Since KIP-1282 couldn't reach a consensus, KIP-1327 >>>>> goes >>>>>> back to using flexible configurations to prevent users from falling >>>> into >>>>>> that pitfall. >>>>>> >>>>>> Best, Chia-Ping >>>>>> >>>>>> Jun Rao via dev <[email protected]> 於 2026年6月13日週六 上午6:49寫道： >>>>>> >>>>>>> Hi, Jiunn-Yang, >>>>>>> >>>>>>> Thanks for the reply and sorry for the late reply. >>>>>>> >>>>>>> JR1. The design of auto.offset.reset.max.age.ms >>>> <https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$> >>>> still feels weird to >>>>> me. >>>>>>> It >>>>>>> categorizes partitions as new or existing based on the partition >>>>> creation >>>>>>> time. Intuitively, the categorization should be based on the group >>>>> creation >>>>>>> time: all partitions existing when the group is created are existing >>>> and >>>>>>> all partitions created after the group creation are new partitions. >>>>>>> >>>>>>> Jun >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 9, 2026 at 8:51 AM 黃竣陽 <[email protected]> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Manually bumping this thread. If there is no further >>>>>>>> discussion, I will close the vote. >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> Jiunn-Yang >>>>>>>> >>>>>>>>> 黃竣陽 <[email protected]> 於 2026年6月1日 晚上7:16 寫道： >>>>>>>>> >>>>>>>>> Hello Jian, >>>>>>>>> >>>>>>>>> Thanks for your feedback, >>>>>>>>> >>>>>>>>> Agreed, partition expansion is a common operational task, not an >>>> edge >>>>>>>>> case. I've updated the Motivation section accordingly. >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> Jiunn-Yang >>>>>>>>> >>>>>>>>>> jian fu <[email protected]> 於 2026年6月1日 下午5:49 寫道： >>>>>>>>>> >>>>>>>>>> Hi Jiunn-Yang: >>>>>>>>>> >>>>>>>>>> Thanks for the KIP. I think it would be useful to clarify that >>>> this >>>>>>> is a >>>>>>>>>> common scenario rather than an edge case, which further >>>> demonstrates >>>>>>> the >>>>>>>>>> need for this optimization. For example: >>>>>>>>>> A partition expansion is a common operational task in Kafka: To >>>>>>> balance >>>>>>>>>> resource utilization and cost, topics are typically created with a >>>>>>>> moderate >>>>>>>>>> default partition count. However, as traffic grows over time, it >>>> is >>>>>>>> often >>>>>>>>>> necessary to increase the number of partitions to accommodate the >>>>>>> higher >>>>>>>>>> workload. >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> Jian >>>>>>>>>> >>>>>>>>>> 黃竣陽 <[email protected]> 于2026年5月30日周六 22:31写道： >>>>>>>>>> >>>>>>>>>>> Hello chia, >>>>>>>>>>> >>>>>>>>>>> Thanks for the comments, I have updated the KIP! >>>>>>>>>>> >>>>>>>>>>> Best Regards, >>>>>>>>>>> Jiunn-Yang >>>>>>>>>>> >>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年5月30日 晚上8:29 寫道： >>>>>>>>>>>> >>>>>>>>>>>> Hi Jiunn-Yang, >>>>>>>>>>>> >>>>>>>>>>>> Would you mind removing the terms "hot" and "cold" when >>>> describing >>>>>>>>>>>> partitions in the KIP? I understand you are using them to >>>> describe >>>>>>> the >>>>>>>>>>>> "freshness" or the users' need for the records, but applying >>>> these >>>>>>>> terms >>>>>>>>>>> to >>>>>>>>>>>> the partition itself feels a bit unnatural. >>>>>>>>>>>> >>>>>>>>>>>> After all, in this scenario, users don't really care whether a >>>>>>>> partition >>>>>>>>>>> is >>>>>>>>>>>> newly expanded or not. Their only expectation is that they won't >>>>>>>> silently >>>>>>>>>>>> lose any live records produced to the topic during their active >>>>>>>>>>> consumption. >>>>>>>>>>>> >>>>>>>>>>>> Best, Chia-Ping >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月30日週六 下午12:30寫道： >>>>>>>>>>>> >>>>>>>>>>>>> Hello Jun, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the feedback, I have updated the KIP motivation >>>>> section. >>>>>>>>>>>>> >>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>> Jiunn-Yang >>>>>>>>>>>>> >>>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年5月30日 凌晨1:12 >>>> 寫道： >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, Jiunn-Yang, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for the reply. I think we need a stronger motivation >>>> for >>>>>>> the >>>>>>>>>>> KIP. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The KIP says "The core insight is that not all partitions >>>> without >>>>>>> a >>>>>>>>>>>>>> committed offset are the same. A newly expanded partition >>>> (hot) >>>>> is >>>>>>>>>>>>>> fundamentally different from a partition the consumer has >>>> never >>>>>>> seen >>>>>>>>>>>>>> because it predates the group (cold)." Why is the hot >>>> partition >>>>>>>>>>>>>> fundamentally different from the cold? >>>>>>>>>>>>>> >>>>>>>>>>>>>> The KIP says "The existing by_duration policy is also >>>>> insufficient >>>>>>>>>>>>> because: >>>>>>>>>>>>>> >>>>>>>>>>>>>> - The calculated seek time (now() - duration) varies across >>>> nodes >>>>>>>> due >>>>>>>>>>>>> to >>>>>>>>>>>>>> clock skew. To be safe, users must set an overly large >>>> duration, >>>>>>>>>>>>> causing >>>>>>>>>>>>>> unnecessary reprocessing. >>>>>>>>>>>>>> - On network errors, the client recalculates the seek time on >>>>>>> retry, >>>>>>>>>>>>>> shifting the target timestamp forward and risking data loss." >>>>>>>>>>>>>> >>>>>>>>>>>>>> However, both of these situations are rare. If these issues >>>>>>> persist, >>>>>>>>>>> more >>>>>>>>>>>>>> severe problems likely exist elsewhere. Rare situations don't >>>>>>> need a >>>>>>>>>>>>> common >>>>>>>>>>>>>> solution. If users care about those rare situations, they can >>>>>>>> implement >>>>>>>>>>>>>> customized logic using >>>>>>>>>>> ConsumerRebalanceListener.onPartitionsAssigned(). >>>>>>>>>>>>>> >>>>>>>>>>>>>> Jun >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sun, May 17, 2026 at 6:50 AM 黃竣陽 <[email protected]> >>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello chia, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for the feedback, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If the creation time exists, the returned value should >>>> always >>>>> be >>>>>>>>>>>>> greater >>>>>>>>>>>>>>> than or equal to zero, right? >>>>>>>>>>>>>>> I have explicitly mentioned this in the KIP. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive any >>>>>>>> field >>>>>>>>>>>>>>> absent UnsupportedVersionException >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The earliest point at which we can detect the version >>>> mismatch >>>>> is >>>>>>>>>>> during >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> first metadata fetch after assignment, which occurs inside >>>>>>> poll(). >>>>>>>>>>>>>>> Therefore, the >>>>>>>>>>>>>>> user would encounter an UnsupportedVersionException from >>>> poll(). >>>>>>>> I’ll >>>>>>>>>>>>>>> clarify this in the KIP. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>> Jiunn-Yang >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年5月17日 下午4:50 >>>> 寫道： >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> hi Jiunn >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> PartitionAgeMs (int64, default -1): The age of this >>>> partition >>>>>>> in >>>>>>>>>>>>>>> milliseconds, computed server-side by the broker as >>>>>>>>>>> broker_current_time >>>>>>>>>>>>> - >>>>>>>>>>>>>>> partition_creation_time. Returns -1 if the broker does not >>>>>>> support >>>>>>>>>>> this >>>>>>>>>>>>>>> feature or the partition creation time is unknown. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If the creation time exists, the returned value should >>>> always >>>>> be >>>>>>>>>>>>> greater >>>>>>>>>>>>>>> than or equal to zero, right? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive any >>>>>>>> field >>>>>>>>>>>>>>> absent UnsupportedVersionException >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Will user encounter UnsupportedVersionException when calling >>>>>>>>>>> `poll()`? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>> Chia-Ping >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2026/05/16 04:30:49 黃竣陽 wrote: >>>>>>>>>>>>>>>>> Hello Jun, chia, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I've updated KIP-1327 with a design change based on the >>>>>>>> discussion >>>>>>>>>>>>>>>>> feedback. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The updated design decouples the new-partition reset >>>> behavior >>>>>>>> from >>>>>>>>>>>>>>>>> the base auto.offset.reset policy: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - auto.offset.reset.max.age.ms >>>> <https://urldefense.com/v3/__http://auto.offset.reset.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpwnwknH$> >>>> now applies to all >>>>>>>> auto.offset.reset >>>>>>>>>>>>>>> values >>>>>>>>>>>>>>>>> (latest, earliest, by_duration, none). >>>>>>>>>>>>>>>>> - For new ("hot") partitions, the consumer resets to >>>>>>>>>>>>>>> auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$> >>>> .partitions >>>>>>>>>>>>>>>>> config setting >>>>>>>>>>>>>>>>> - For existing ("cold") partitions, the base >>>> auto.offset.reset >>>>>>>>>>> policy >>>>>>>>>>>>>>> continues >>>>>>>>>>>>>>>>> to apply unchanged. >>>>>>>>>>>>>>>>> - The new-partition reset behavior is represented by a >>>>> separate >>>>>>>>>>>>>>> internal config >>>>>>>>>>>>>>>>> (auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$>.partitions, >>>> currently fixed to >>>>>>> earliest). >>>>>>>>>>> This >>>>>>>>>>>>>>> decoupled design makes >>>>>>>>>>>>>>>>> it straightforward to promote the behavior to a public >>>>>>>> user-facing >>>>>>>>>>>>>>> configuration in a future KIP. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>> Jiunn-Yang >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年5月16日 清晨7:46 >>>> 寫道： >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> hi Jun >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I see what you mean now. The proposal from me is listed >>>>> below: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1) Add auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$>.partitions >>>> with a default value >>>>>>> of >>>>>>>>>>>>>>> earliest. It fixes the data loss from both by_duration and >>>>>>> latest, >>>>>>>> and >>>>>>>>>>>>> it >>>>>>>>>>>>>>> does not change the logic of auto.offset.reset=earliest. >>>>>>>>>>>>>>>>>> 2) Mark auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$>.partitions >>>> as an internal >>>>>>>>>>>>>>> configuration. auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$> >>>> .partitions=earliest >>>>> already >>>>>>>>>>>>>>> addresses the issue, and we can discuss the use cases of >>>> other >>>>>>>> values >>>>>>>>>>>>> in a >>>>>>>>>>>>>>> separate KIP. >>>>>>>>>>>>>>>>>> 3) Both configs, auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$>.partitions >>>> and >>>>>>>>>>>>>>> auto.offset.reset.latest.max.age.ms >>>> <https://urldefense.com/v3/__http://auto.offset.reset.latest.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfu9JSP4l$>, >>>> will be applied to all for >>>>>>>>>>>>>>> consistency. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> WDYT? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 2026/05/15 20:53:20 Jun Rao via dev wrote: >>>>>>>>>>>>>>>>>>> Hi, Chia-Ping, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks for the reply. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 1. In the motivation section, the KIP says "When a Kafka >>>>>>> topic >>>>>>>> is >>>>>>>>>>>>>>> expanded >>>>>>>>>>>>>>>>>>> with new partitions, consumers using the latest auto >>>> offset >>>>>>>> reset >>>>>>>>>>>>>>> policy >>>>>>>>>>>>>>>>>>> will silently miss all records produced to those >>>> partitions >>>>>>>> before >>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> consumer discovers them.". If a user sets >>>>>>>>>>>>>>>>>>> auto.offset.reset=by_duration=1sec, the same record loss >>>>>>> issue >>>>>>>>>>> could >>>>>>>>>>>>>>> also >>>>>>>>>>>>>>>>>>> happen, right? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 2. I was thinking auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$>.partitions >>>> will >>>>> take >>>>>>>> the >>>>>>>>>>>>> same >>>>>>>>>>>>>>>>>>> values as auto.offset.reset. So a user could set it >>>>>>>> by_duration if >>>>>>>>>>>>>>> needed. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Jun >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Thu, May 14, 2026 at 4:06 PM Chia-Ping Tsai < >>>>>>>>>>> [email protected] >>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> hi Jun >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks for the feedback. I might be missing something >>>>>>>> important >>>>>>>>>>>>> from >>>>>>>>>>>>>>> your >>>>>>>>>>>>>>>>>>>> suggestion, so please bear with me as I try to clarify >>>> with >>>>>>> a >>>>>>>> few >>>>>>>>>>>>>>> questions: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1. Is there a strong use case for extending this logic >>>> to >>>>>>>> other >>>>>>>>>>>>> reset >>>>>>>>>>>>>>>>>>>> policies? Unlike latest, policies like earliest or >>>>>>> by_duration >>>>>>>>>>>>> don't >>>>>>>>>>>>>>> seem >>>>>>>>>>>>>>>>>>>> to suffer from the same silent data loss issue when a >>>>>>>> partition >>>>>>>>>>> is >>>>>>>>>>>>>>> expanded. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 2. What values would we expect users to configure for >>>>>>>>>>>>>>>>>>>> auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$>.partitions? >>>> If they set it to >>>>>>> earliest >>>>>>>> or >>>>>>>>>>>>>>> latest, >>>>>>>>>>>>>>>>>>>> we might run into the exact same edge cases. For >>>> example, >>>>>>> if a >>>>>>>>>>>>>>> consumer is >>>>>>>>>>>>>>>>>>>> offline for a while and a new partition is created >>>> during >>>>>>> that >>>>>>>>>>>>>>> downtime, >>>>>>>>>>>>>>>>>>>> the user might actually want to skip to latest when >>>>>>> resuming, >>>>>>>>>>>>> rather >>>>>>>>>>>>>>> than >>>>>>>>>>>>>>>>>>>> reading from earliest just because the partition is >>>>>>>> technically >>>>>>>>>>>>>>> "new" to >>>>>>>>>>>>>>>>>>>> the group. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This is exactly why we opted for introducing a max.age >>>>>>>> threshold. >>>>>>>>>>>>> It >>>>>>>>>>>>>>> gives >>>>>>>>>>>>>>>>>>>> users a time-bound way to define what is genuinely >>>>> "hot/new" >>>>>>>> and >>>>>>>>>>>>>>> what is >>>>>>>>>>>>>>>>>>>> just an old partition they haven't seen yet. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>> Chia-Ping >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 2026/05/14 20:48:09 Jun Rao via dev wrote: >>>>>>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I find auto.offset.reset.latest.max.age a bit weird. It >>>>>>> only >>>>>>>>>>>>>>> applies when >>>>>>>>>>>>>>>>>>>>> auto.offset.reset is latest. However, it seems that the >>>>>>>>>>> motivation >>>>>>>>>>>>>>>>>>>> equally >>>>>>>>>>>>>>>>>>>>> applies when auto.offset.reset is set to other values >>>> like >>>>>>>>>>>>>>> by_duration. >>>>>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>>> intention is that we want to have a separate way to >>>>> control >>>>>>>>>>> newly >>>>>>>>>>>>>>> created >>>>>>>>>>>>>>>>>>>>> partitions vs existing partitions when the group >>>> starts. >>>>>>>> Have we >>>>>>>>>>>>>>>>>>>> considered >>>>>>>>>>>>>>>>>>>>> adding a new config like auto.offset.reset.new >>>> <https://urldefense.com/v3/__http://auto.offset.reset.new__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkfpDMjdw3$> >>>>> .partitions? >>>>>>>> If >>>>>>>>>>>>> this >>>>>>>>>>>>>>> new >>>>>>>>>>>>>>>>>>>>> config is not set, the offset reset policy defaults to >>>> the >>>>>>>>>>> policy >>>>>>>>>>>>>>> used >>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>> existing partitions. The user could set it explicitly >>>> to >>>>>>>>>>> customize >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>> behavior for new partitions. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Jun >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Thu, May 7, 2026 at 5:07 AM 黃竣陽 <[email protected] >>>>> >>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I’d like to manually bump this thread. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月1日 晚上10:37 寫道： >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hello all, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> DJ01/DJ02: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> MetadataResponse bumps from v13 to v14. The >>>>>>>> PartitionMetadata >>>>>>>>>>>>>>> struct >>>>>>>>>>>>>>>>>>>>>> gains a new >>>>>>>>>>>>>>>>>>>>>>> field PartitionAgeMs (int64, default -1), computed >>>>>>>> server-side >>>>>>>>>>>>> by >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> broker as >>>>>>>>>>>>>>>>>>>>>>> broker_current_time - partition_creation_time. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Also add the consumer heartbeat flow. when >>>>>>>> MembershipManager >>>>>>>>>>>>>>> detects >>>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>> newly assigned >>>>>>>>>>>>>>>>>>>>>>> partition, it explicitly invalidates the metadata for >>>>> the >>>>>>>>>>>>> affected >>>>>>>>>>>>>>>>>>>> topic >>>>>>>>>>>>>>>>>>>>>> and forces a fresh MetadataRequest >>>>>>>>>>>>>>>>>>>>>>> before making the offset reset decision, even if the >>>>>>> topic >>>>>>>> ID >>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>> already >>>>>>>>>>>>>>>>>>>>>> in the cache. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> MB0: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The consumer learns the broker's maximum supported >>>>>>>>>>>>>>> MetadataResponse >>>>>>>>>>>>>>>>>>>>>> version via the >>>>>>>>>>>>>>>>>>>>>>> ApiVersions negotiation at connection time. If the >>>>>>>> negotiated >>>>>>>>>>>>>>>>>>>> version is >>>>>>>>>>>>>>>>>>>>>> unsupported, the consumer >>>>>>>>>>>>>>>>>>>>>>> knows the broker does not support PartitionAgeMs at >>>> all >>>>>>> and >>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>> throw an >>>>>>>>>>>>>>>>>>>>>> UnsupportedVersionException >>>>>>>>>>>>>>>>>>>>>>> immediately, rather than silently falling back to >>>> latest >>>>>>>> and >>>>>>>>>>>>>>> risking >>>>>>>>>>>>>>>>>>>>>> data loss without any operator-visible signal. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> MB1/MB2/MB3: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I have addressed these changes in the KIP. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年4月29日 >>>>> 下午4:04 >>>>>>>> 寫道： >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> hi David >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I agree with the direction of moving the 'age' >>>>>>> resolution >>>>>>>>>>> from >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> Heartbeat API to the Metadata API to keep the control >>>>>>> plane >>>>>>>>>>>>> clean. >>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>> main >>>>>>>>>>>>>>>>>>>>>> trade-off, as we noted before, is introducing >>>>> inter-broker >>>>>>>>>>> clock >>>>>>>>>>>>>>> skew. >>>>>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>>>> Group Coordinator approach provided a single source of >>>>>>> truth >>>>>>>>>>> for >>>>>>>>>>>>>>> time. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> However, realistically, this time skew should be >>>>>>>> negligible. >>>>>>>>>>>>>>> Given >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>> the max.age threshold will likely be configured in >>>>> minutes >>>>>>>> or >>>>>>>>>>>>>>> hours, a >>>>>>>>>>>>>>>>>>>>>> typical NTP skew (in milliseconds) between brokers >>>> won't >>>>>>>> impact >>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> fallback decision. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> David Jacot via dev <[email protected]> 於 >>>>>>> 2026年4月29日 >>>>>>>>>>>>> 下午3:29 >>>>>>>>>>>>>>> 寫道： >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP! >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Sorry, I haven't really followed the previous >>>>>>>> conversation >>>>>>>>>>>>> but I >>>>>>>>>>>>>>>>>>>> took a >>>>>>>>>>>>>>>>>>>>>>>>> quick look at this one. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> DJ01: I don't clearly understand the flow with the >>>>>>>>>>>>>>>>>>>>>> ConsumerGroupHeartbeat >>>>>>>>>>>>>>>>>>>>>>>>> API after reading the KIP. There is a new boolean; >>>> the >>>>>>>> KIP >>>>>>>>>>>>>>> states >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>> partition ages are returned only when this boolean >>>> is >>>>>>>> set. >>>>>>>>>>>>>>>>>>>> Implicitly, >>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>> means that when the consumer receives a new >>>> partition, >>>>>>> it >>>>>>>>>>> will >>>>>>>>>>>>>>>>>>>> issue a >>>>>>>>>>>>>>>>>>>>>> new >>>>>>>>>>>>>>>>>>>>>>>>> HB request with the boolean set to receive the >>>> ages. >>>>> Is >>>>>>>> my >>>>>>>>>>>>>>>>>>>>>> understanding >>>>>>>>>>>>>>>>>>>>>>>>> correct? We should perhaps clarify the flow and >>>> also >>>>>>>> explain >>>>>>>>>>>>>>> how it >>>>>>>>>>>>>>>>>>>>>> fits >>>>>>>>>>>>>>>>>>>>>>>>> into the existing flow (e.g. list offsets, fetch >>>>>>> offsets, >>>>>>>>>>>>> etc.). >>>>>>>>>>>>>>>>>>>>>>>>> DJ02: It my understanding is correct, I wonder if >>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API is the right place >>>> for >>>>>>>> this >>>>>>>>>>>>> given >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>> a new >>>>>>>>>>>>>>>>>>>>>>>>> round trip is done anyway. Alternatively, it could >>>>>>> simply >>>>>>>>>>>>>>> include >>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>> metadata. Generally, we should be rather cautious >>>>> about >>>>>>>> not >>>>>>>>>>>>>>>>>>>> overloading >>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API with unrelated >>>>> concepts. >>>>>>>> The >>>>>>>>>>>>> API >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>> control plane API for assigning or revoking >>>>> partitions. >>>>>>>> The >>>>>>>>>>>>> fact >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>>> don't want to add it to the corresponding Streams >>>> API >>>>>>>> also >>>>>>>>>>>>>>> suggests >>>>>>>>>>>>>>>>>>>>>>>>> something is not quite right. What would we do if >>>> we >>>>>>>> want to >>>>>>>>>>>>>>>>>>>> support >>>>>>>>>>>>>>>>>>>>>>>>> Streams in the future? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Apr 29, 2026 at 12:28 AM Muralidhar Basani >>>>> via >>>>>>>> dev >>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jiunn, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for this great kip. Good to know about >>>> the >>>>>>>> gap. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> mb-0 - why a new v2 version bump for >>>>>>>> RequestPartitionAges >>>>>>>>>>>>>>> field. >>>>>>>>>>>>>>>>>>>> Can a >>>>>>>>>>>>>>>>>>>>>>>>>> tagged field (for ex: on response, PartitionAges >>>> on >>>>>>>>>>>>>>>>>>>> TopicPartitions) >>>>>>>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>>> used here and avoid version bump? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> mb-1 - For the new config, is there a recommended >>>>>>> value >>>>>>>> or >>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>> ConfigDef >>>>>>>>>>>>>>>>>>>>>>>>>> validator? Probably it should based on the >>>>>>>>>>>>> metadata.max.age.ms >>>> <https://urldefense.com/v3/__http://metadata.max.age.ms__;!!Ayb5sqE7!ryUSIElKDF-DJJHgYwYXwp4XEBXpXuBOnZd18PJoMNH4LZ1gc-pDbbdfb2eme_dRSvdvI3bkflKEb5SK$> >>>>>>>>>>>>>>> ? >>>>>>>>>>>>>>>>>>>>>> Sizing >>>>>>>>>>>>>>>>>>>>>>>>>> instructions can be part of javadocs I guess. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> mb-2 - (minor) As there are no changes to Kafka >>>>>>> Streams, >>>>>>>>>>>>> would >>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>> better >>>>>>>>>>>>>>>>>>>>>>>>>> to add this new config >>>>>>> auto.offset.reset.latest.max.age >>>>>>>> to >>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>> StreamsConfig block list >>>>>>>>>>>>>>>>>>>> (NON_CONFIGURABLE_CONSUMER_DEFAULT_CONFIGS) >>>>>>>>>>>>>>>>>>>>>> for a >>>>>>>>>>>>>>>>>>>>>>>>>> clear warning, incase users configure it? This is >>>> the >>>>>>>> most >>>>>>>>>>>>>>>>>>>> familiar >>>>>>>>>>>>>>>>>>>>>>>>>> consumer config and users might easily mistakenly >>>>>>>> configure >>>>>>>>>>>>>>> it. Or >>>>>>>>>>>>>>>>>>>>>> may be >>>>>>>>>>>>>>>>>>>>>>>>>> it's not worth it to add. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> mb-3 - (minor) The phrasing "the consumer falls >>>> back >>>>>>> to >>>>>>>>>>>>>>> earliest" >>>>>>>>>>>>>>>>>>>>>> reads as >>>>>>>>>>>>>>>>>>>>>>>>>> if the config were being changed per-partition >>>> which >>>>>>>> isn't >>>>>>>>>>>>>>>>>>>> supported. >>>>>>>>>>>>>>>>>>>>>> May >>>>>>>>>>>>>>>>>>>>>>>>>> be rephrasing to something like "consumer resolves >>>>> the >>>>>>>>>>>>> initial >>>>>>>>>>>>>>>>>>>>>> position to >>>>>>>>>>>>>>>>>>>>>>>>>> start offset for that partition" as if earliest >>>> was >>>>>>>> applied >>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>> partition only and auto.offset.reset config is >>>>>>>> unchanged. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> Murali >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Apr 28, 2026 at 2:48 PM 黃竣陽 < >>>>>>>> [email protected]> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi chia, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I have updated the KIP to include this change. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 >>>> 2026年4月28日 >>>>>>>> 晚上8:03 >>>>>>>>>>>>> 寫道： >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jiunn-Yang >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> chia_0: Should we expose the partition creation >>>>> time >>>>>>>> via >>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>> Admin >>>>>>>>>>>>>>>>>>>>>> API? >>>>>>>>>>>>>>>>>>>>>>>>>>> I assume it would be valuable for users to >>>> diagnose >>>>>>> and >>>>>>>>>>>>>>>>>>>> troubleshoot >>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>> behavior of auto.offset.reset.latest.max.age >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/04/28 10:47:58 黃竣陽 wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to start a discussion on KIP-1327 >>>>>>>> Prevent >>>>>>>>>>> Hot >>>>>>>>>>>>>>> Data >>>>>>>>>>>>>>>>>>>>>> Loss >>>>>>>>>>>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>>>>>>>> Partition Expansion for Latest Policy >>>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/KY4mGQ__;!!Ayb5sqE7!qF4q1QzF1RRgP61D7A2xuEai1ky7fepKDKFFvpNBuePikH-ULmT87TvuuZzy5kau5E4y5zMZAmfQQiwZomM$ >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This proposal aims to introduces >>>>>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age, >>>>>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>>>> consumer config that lets the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest reset policy distinguish newly expanded >>>>>>> (hot) >>>>>>>>>>>>>>> partitions >>>>>>>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>>>>>>>>>>> long-existing (cold) ones. Partitions >>>>>>>>>>>>>>>>>>>>>>>>>>>>> younger than the configured threshold >>>>> automatically >>>>>>>> fall >>>>>>>>>>>>>>> back >>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>> earliest, preventing silent data loss >>>>>>>>>>>>>>>>>>>>>>>>>>>>> during topic expansion without forcing a full >>>>>>>> historical >>>>>>>>>>>>>>>>>>>> reprocess. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>>
