Hey, I have made the changes in the PR https://github.com/apache/kafka/pull/21468 Please review the changes once you get some time.
- Manan On Mon, Feb 2, 2026 at 5:41 PM Manan Gupta <[email protected]> wrote: > Hi Luke, > > I have updated the Rejected Alternatives section. > Also, I have initiated the voting process for the KIP. > https://lists.apache.org/thread/zsfqfwz56b7pmtw5lt7tr3lk200mtwqy > > - Manan > > On Mon, Feb 2, 2026 at 5:08 PM Luke Chen <[email protected]> wrote: > >> Hi Manan, >> >> Thanks for the update. >> >> About the time-based retention metrics, I agree with your thoughts. >> But I think we should put it into the Rejected Alternatives section for >> future reference. >> >> Otherwise, LGTM! >> >> Luke >> >> >> On Mon, Feb 2, 2026 at 3:25 PM Manan Gupta <[email protected]> wrote: >> >> > Hi Luke, >> > >> > Thanks for the thoughtful feedback — I appreciate you taking a close >> look >> > at the KIP. >> > >> > On the naming point, I agree that SizeInPercent can be ambiguous and >> might >> > be interpreted as disk utilization. Following your suggestion, I’ve >> updated >> > the KIP to use *RetentionSizeInPercent*, which more clearly communicates >> > that the metric represents a partition’s log size relative to its >> > configured retention limit rather than physical disk capacity. >> > Additionally, the metric is scoped using topic and partition tags. This >> > allows for strong correlation with other per-partition metrics and makes >> > the context of the value explicit. >> > >> > Regarding a retention-time–based metric (for example, exposing the time >> > until the oldest segment expires), I considered this but am not >> convinced >> > it would be particularly actionable in practice. For topics older than >> the >> > configured retention time that have ongoing production, log segments >> tend >> > to remain near expiration continuously. This would cause such a metric >> to >> > hover close to zero in steady state. As a result, it may not provide >> > meaningful operational signal to topic owners, since data older than the >> > retention time is expected to be eligible for deletion. >> > >> > For this reason, the current KIP focuses on size-based retention >> metrics, >> > which more directly indicate proximity to retention-triggered cleanup >> due >> > to storage pressure. I’m happy to revisit time-based metrics separately >> if >> > there are concrete use cases where they would add clear operational >> value. >> > >> > Thanks again for the suggestions — please let me know if you see any >> other >> > areas that could benefit from clarification. >> > >> > >> > On Fri, Jan 16, 2026 at 3:49 PM Luke Chen <[email protected]> wrote: >> > >> > > Hi Manan, >> > > >> > > Thanks for the KIP. >> > > >> > > 1. I agree this is a good improvement, but the naming is not clear >> IMO. >> > > "SizeInPercent" makes me think the disk is going to be full after it's >> > > 100%. >> > > Maybe "RetentionSizeInPercent"? >> > > >> > > 2. Do we need the similar metrics for the time retention? >> > > Like "RetentionTimeInSec", which is to show the time gap between >> oldest >> > > segment with the retention time, "RetentionTimeInSec = 300" means the >> > > oldest segment will be expired after 300 seconds. Is that useful? >> > > >> > > >> > > Thank you, >> > > Luke >> > > >> > > On Mon, Jan 12, 2026 at 6:17 PM Manan Gupta <[email protected]> >> > wrote: >> > > >> > > > Gentle reminder for feedback on the KIP-1257: Partition Size >> Percentage >> > > > Metrics for Storage Monitoring proposal. >> > > > >> > > > On Tue, Dec 16, 2025 at 5:34 PM Manan Gupta <[email protected]> >> > > wrote: >> > > > >> > > > > Hi all, >> > > > > >> > > > > This email starts the discussion thread for KIP-1257: Partition >> Size >> > > > > Percentage Metrics for Storage Monitoring. This KIP introduces >> > > > > retention-aware, percentage-based partition metrics that >> > significantly >> > > > > improve Kafka’s storage observability. The proposed metrics >> simplify >> > > > > alerting, enhance capacity planning, and provide clear visibility >> > into >> > > > > retention pressure—especially for tiered storage—while remaining >> > > > > lightweight, backward compatible, and operationally intuitive. >> > > > > >> > > > > I'd appreciate your initial thoughts and feedback on the proposal. >> > > > > https://cwiki.apache.org/confluence/x/MAEXG >> > > > > >> > > > > >> > > > > Thanks, >> > > > > Manan >> > > > > >> > > > >> > > >> > >> >
