Hi Chia, Thanks for taking another look. From what I understand, pruning has to wait for the slowest group on the state partition to snapshot before it can delete old records.
For ex: if groups A and B share a state partition: - A snapshots frequently .. its latest snapshot is at offset 900. - B has low traffic ... its latest snapshot is still at offset 100. So the prune point is min(900, 100) which is 100, so only offsets 0–99 can be deleted. Everything A writes after that stays on disk until B snapshots again. So B's threshold is controlling the size than A's snapshots, whether it's in bytes or a count. I had moved the KIP to bytes after your earlier suggestion (chia_01), but I agree the pruning behaviour makes bytes much less useful here than in a buffering case like KIP-1325, where a byte limit directly bounds memory. I think we had earlier calculations showed update records are a fairly constant size, so a count gives nearly the same result. I shall revise the KIP back to a count based approach and follow up here. Thanks, Murali On Tue, Oct 6, 2026 at 8:52 AM Chia-Ping Tsai <[email protected]> wrote: > Hi, > > After re-reading the KIP, I'm not sure whether bytes is better than count. > Eventually the prune must wait for "all" share partitions on the state > partition to be snapshotted. That means users can hardly evaluate the > retained log size from a per-partition threshold, no matter it is count or > bytes. > > On the other side, the concept of "per-group" is good to me, since it > allows the low-traffic groups to snapshot more often. > > So I'm wondering whether a per-group count config (plus raising the broker > upper bound) is enough for this KIP. > > Best, > Chia-Ping > > On 2026-05-22 19:30 UTC Muralidhar Basani via dev wrote: > > Hi all, > > > > I would like to start a discussion on KIP-1349, which allows configurable > > snapshot frequency of share groups. > > > > KIP : > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1349%3A+Configurable+snapshot+frequency+for+share+groups > > > > Thanks, > > Murali > > >
