Hi Murali, Would you mind updating KIP-1349 according to our discussion, i.e. switching back to a per-group count config and raising the broker-level upper bound?
Also, the KIP is inconsistent about how a group-level value above the broker-level limit is handled: Public Interfaces says it's rejected at alter time, while Proposed Changes says it's clamped. Since the broker config can change after the group config is set, I think we should clamp the value at runtime. Once the KIP is updated, it may be worth restarting the vote. Best, Chia-Ping On Thu, Oct 08, 2026 02:25 PM, Muralidhar Basani via dev < [email protected]> wrote: > Hi Chia, > Thanks for taking another look. From what I understand, pruning has to wait > for the slowest group on the state partition to snapshot before it can > delete old records. > > For ex: if groups A and B share a state partition: > - A snapshots frequently .. its latest snapshot is at offset 900. > - B has low traffic ... its latest snapshot is still at offset 100. > > So the prune point is min(900, 100) which is 100, so only offsets 0–99 can > be deleted. Everything A writes after that stays on disk until B snapshots > again. So B's threshold is controlling the size than A's snapshots, whether > it's in bytes or a count. > > I had moved the KIP to bytes after your earlier suggestion (chia_01), but I > agree the pruning behaviour makes bytes much less useful here than in a > buffering case like KIP-1325, where a byte limit directly bounds memory. > > I think we had earlier calculations showed update records are a fairly > constant size, so a count gives nearly the same result. > > I shall revise the KIP back to a count based approach and follow up here. > > Thanks, > Murali > > On Tue, Oct 6, 2026 at 8:52 AM Chia-Ping Tsai <[email protected]> wrote: > > > Hi, > > > > After re-reading the KIP, I'm not sure whether bytes is better than > count. > > Eventually the prune must wait for "all" share partitions on the state > > partition to be snapshotted. That means users can hardly evaluate the > > retained log size from a per-partition threshold, no matter it is count > or > > bytes. > > > > On the other side, the concept of "per-group" is good to me, since it > > allows the low-traffic groups to snapshot more often. > > > > So I'm wondering whether a per-group count config (plus raising the > broker > > upper bound) is enough for this KIP. > > > > Best, > > Chia-Ping > > > > On 2026-05-22 19:30 UTC Muralidhar Basani via dev wrote: > > > Hi all, > > > > > > I would like to start a discussion on KIP-1349, which allows > configurable > > > snapshot frequency of share groups. > > > > > > KIP : > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1349% > 3A+Configurable+snapshot+frequency+for+share+groups > > > > > > Thanks, > > > Murali > > > > > >
