Hi all,

I have updated the KIP based on the feedback so far:

- Compatibility: added a "Disk sizing" section. For backfill data, local
disk grows by about local.retention.ms * write rate per partition replica,
with local.retention.bytes as the way to cap it.
- Proposed Changes: clarified where the upload time comes from, and that
local time-based deletion pauses while the remote log metadata cache is
initializing, with no fallback to record timestamps.
- Rejected Alternatives: added min(record timestamp, upload time).

Please take a look and let me know if there are further comments.

Thanks,
Murali

On Fri, Nov 13, 2026 at 10:29 PM Muralidhar Basani <
[email protected]> wrote:

> Hi Viktor,
>
> Thanks for the review, and nice point on disk sizing.
>
> Unless there are objections, I shall keep the uploadTime as base, add the
> sizing note to compatibility, and move min(..) to rejected alternatives.
>
> I will update KIP.
>
> Chia, pls let me know if this addresses the concern you raised on Case C.
>
> Thanks,
> Murali
>
> On Thu, Nov 12, 2026 at 11:32 AM Viktor Somogyi-Vass <
> [email protected]> wrote:
>
>> Hi Murali,
>>
>> I also agree with Chia-Ping, taking uploadTime as the new base for
>> local.retention.ms would generally increase local disk usage which might
>> be surprising and might have disk sizing consequences as well. So the
>> extra local disk required would be local.retention.ms * upload rate, I
>> think we can add this to the KIP. I don't think that changing behavior is a
>> blocker if it results in a more predictable behavior.
>> I was also considering min(maxTimestamp, uploadTime), but I think that
>> overall it would complicate the config itself, which I would not take
>> compared to uploadTime only as that would create a common, broker-bound and
>> predictable interpretation for local.retention.ms.
>>
>> Best,
>> Viktor
>>
>> On Thu, Nov 5, 2026 at 10:03 PM Muralidhar Basani via dev <
>> [email protected]> wrote:
>>
>>> Thanks Chia for the review.
>>>
>>> 1. Case C: Yes, this is a behavior change. It only affects topics that
>>> set
>>> local.retention.ms lower than the age of the backfilled data.
>>> With the default (-2 for local.retention.ms, means use the same value as
>>> retention.ms), backfilled data isn't deleted right after upload today
>>> either.
>>> For example, with retention.ms = 7 days and local.retention.ms left at
>>> its
>>> default of -2, local retention is also 7 days. Data stays on local disk
>>> as
>>> long as it's kept at all.
>>>
>>> I see 2 options.
>>>
>>> a. Keep upload-time semantics and document the change in the upgrade
>>> notes.
>>> Operators who want to bound local disk can set local.retention.bytes: it
>>> is
>>> per partition, ignores timestamps, and deletes the oldest uploaded
>>> segments
>>> once the limit is exceeded. For a one-off backfill, they can also lower
>>> local.retention.ms on the topic temporarily.
>>>
>>> Ex for bytes : As the limit applies to each partition separately, with
>>> 5gb
>>> per partition, and with 10 partitions, broker can hold upto 50gb locally.
>>> As it only looks at size, it will ignore the timestamps then.
>>>
>>> b. What do you think of an alternative which is to measure from
>>> min(record
>>> timestamp, upload time). That keeps today's backfill behavior and still
>>> fixes future timestamps, but local.retention.ms would no longer mean the
>>> same thing for all data.
>>>
>>> Which one do you think fits operators better?
>>>
>>> 2. Metadata cache initializing: No, we won't fall back. RLM tasks skip
>>> their run until RemoteLogMetadataManager#isReady is true, so
>>> highestOffsetInRemoteStorage stays -1 and the upload-before-delete guard
>>> blocks time-based local deletion.
>>> Basically deletion is paused until cache is ready. So the fallback isn't
>>> triggered.
>>>
>>> Pls let me know your thoughts.
>>>
>>> Thanks,
>>> Murali
>>>
>>> On Thu, Nov 5, 2026 at 7:06 PM Chia-Ping Tsai via dev <
>>> [email protected]>
>>> wrote:
>>>
>>> > Hi Murali,
>>> >
>>> > I'm worried about Case C: "the local copy is deleted around 11:05, 1
>>> hour
>>> > after the upload."
>>> >
>>> > Would this surprise operators who assume backfilled data is removed
>>> right
>>> > after upload? I guess the suitable approach for them is to use
>>> > local.retention.bytes?
>>> >
>>> > Also, what happens if the RemoteLogMetadataCache is still initializing
>>> > when the broker has just started? Will we always take the fallback?
>>> >
>>> > Best,
>>> > Chia-Ping
>>> >
>>> >
>>> >
>>> > On 2026-10-23 18:30 UTC Muralidhar Basani via dev wrote:
>>> > > Hi all,
>>> > >
>>> > > I would like to start a discussion about tiered storage topics and
>>> how
>>> > > local segments are deleted by time, once they are uploaded to remote
>>> > > storage.
>>> > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/spaces/KAFKA/pages/451979130/KIP-1385+Measure+local.retention.ms+from+remote+upload+time+instead+of+record+timestamps
>>> > >
>>> > > Looking forward to your thoughts on this approach.
>>> > >
>>> > > Thanks,
>>> > > Murali
>>> > >
>>> >
>>>
>>

Reply via email to