Hi Viktor,

Thanks for the review, and nice point on disk sizing.

Unless there are objections, I shall keep the uploadTime as base, add the
sizing note to compatibility, and move min(..) to rejected alternatives.

I will update KIP.

Chia, pls let me know if this addresses the concern you raised on Case C.

Thanks,
Murali

On Thu, Nov 12, 2026 at 11:32 AM Viktor Somogyi-Vass <
[email protected]> wrote:

> Hi Murali,
>
> I also agree with Chia-Ping, taking uploadTime as the new base for
> local.retention.ms would generally increase local disk usage which might
> be surprising and might have disk sizing consequences as well. So the
> extra local disk required would be local.retention.ms * upload rate, I
> think we can add this to the KIP. I don't think that changing behavior is a
> blocker if it results in a more predictable behavior.
> I was also considering min(maxTimestamp, uploadTime), but I think that
> overall it would complicate the config itself, which I would not take
> compared to uploadTime only as that would create a common, broker-bound and
> predictable interpretation for local.retention.ms.
>
> Best,
> Viktor
>
> On Thu, Nov 5, 2026 at 10:03 PM Muralidhar Basani via dev <
> [email protected]> wrote:
>
>> Thanks Chia for the review.
>>
>> 1. Case C: Yes, this is a behavior change. It only affects topics that set
>> local.retention.ms lower than the age of the backfilled data.
>> With the default (-2 for local.retention.ms, means use the same value as
>> retention.ms), backfilled data isn't deleted right after upload today
>> either.
>> For example, with retention.ms = 7 days and local.retention.ms left at
>> its
>> default of -2, local retention is also 7 days. Data stays on local disk as
>> long as it's kept at all.
>>
>> I see 2 options.
>>
>> a. Keep upload-time semantics and document the change in the upgrade
>> notes.
>> Operators who want to bound local disk can set local.retention.bytes: it
>> is
>> per partition, ignores timestamps, and deletes the oldest uploaded
>> segments
>> once the limit is exceeded. For a one-off backfill, they can also lower
>> local.retention.ms on the topic temporarily.
>>
>> Ex for bytes : As the limit applies to each partition separately, with 5gb
>> per partition, and with 10 partitions, broker can hold upto 50gb locally.
>> As it only looks at size, it will ignore the timestamps then.
>>
>> b. What do you think of an alternative which is to measure from min(record
>> timestamp, upload time). That keeps today's backfill behavior and still
>> fixes future timestamps, but local.retention.ms would no longer mean the
>> same thing for all data.
>>
>> Which one do you think fits operators better?
>>
>> 2. Metadata cache initializing: No, we won't fall back. RLM tasks skip
>> their run until RemoteLogMetadataManager#isReady is true, so
>> highestOffsetInRemoteStorage stays -1 and the upload-before-delete guard
>> blocks time-based local deletion.
>> Basically deletion is paused until cache is ready. So the fallback isn't
>> triggered.
>>
>> Pls let me know your thoughts.
>>
>> Thanks,
>> Murali
>>
>> On Thu, Nov 5, 2026 at 7:06 PM Chia-Ping Tsai via dev <
>> [email protected]>
>> wrote:
>>
>> > Hi Murali,
>> >
>> > I'm worried about Case C: "the local copy is deleted around 11:05, 1
>> hour
>> > after the upload."
>> >
>> > Would this surprise operators who assume backfilled data is removed
>> right
>> > after upload? I guess the suitable approach for them is to use
>> > local.retention.bytes?
>> >
>> > Also, what happens if the RemoteLogMetadataCache is still initializing
>> > when the broker has just started? Will we always take the fallback?
>> >
>> > Best,
>> > Chia-Ping
>> >
>> >
>> >
>> > On 2026-10-23 18:30 UTC Muralidhar Basani via dev wrote:
>> > > Hi all,
>> > >
>> > > I would like to start a discussion about tiered storage topics and how
>> > > local segments are deleted by time, once they are uploaded to remote
>> > > storage.
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/spaces/KAFKA/pages/451979130/KIP-1385+Measure+local.retention.ms+from+remote+upload+time+instead+of+record+timestamps
>> > >
>> > > Looking forward to your thoughts on this approach.
>> > >
>> > > Thanks,
>> > > Murali
>> > >
>> >
>>
>

Reply via email to