Hi everyone, Bumping this thread again. I know we're in the middle of the 4.0.2/4.1.2 code freezes and the 4.2 stabilization crunch, but I’d appreciate an initial look at KIP-1267 when things settle down.
The motivation is solving the visibility gap for remote fetch costs. In multi-tenant environments, we can't currently attribute S3/GCS egress to specific consumers because metrics are only at the topic level. This makes chargeback and identifying "noisy neighbors" a pain for anyone running tiered storage at scale. Satish, Harsha, Josep — given your work on KIP-405, I’d specifically value your take on adding client-id tags to the RemoteLogManager. Also interested in whether we should align these with the OpenTelemetry/KIP-714 standards or stick to standard JMX tags. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1267%3A+Tiered+Storage+Cost+Attribution+Metrics Jira: https://issues.apache.org/jira/browse/KAFKA-20047 Regards, Viquar khan https://www.linkedin.com/in/vaquar-khan-b695577/ On Mon, 2 Feb 2026 at 00:35, vaquar khan <[email protected]> wrote: > Hi everyone, > > I am looking for feedback on *KIP-1267 > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1267%3A+Tiered+Storage+Cost+Attribution+Metrics>*. > I’d appreciate it if anyone could take a look and share their thoughts. > > Regards, > > Viquar khan > > https://www.linkedin.com/in/vaquar-khan-b695577/ > > > On Wed, 7 Jan 2026 at 01:11, vaquar khan <[email protected]> wrote: > >> Hi everyone, >> >> I would like to start the discussion for *KIP-1267*: Tiered Storage Cost >> Attribution Metrics. >> >> *KIP:* >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1267%3A+Tiered+Storage+Cost+Attribution+Metrics >> >> >> *Jira:*https://issues.apache.org/jira/browse/KAFKA-20047 >> >> Tiered Storage (KIP-405) has been a great addition for managing storage >> costs, but it introduces variable costs for data access (GET requests and >> egress). Currently, existing metrics only track remote fetches at the topic >> level. >> >> In a multi-tenant cluster, if a specific consumer group triggers a >> massive amount of historical reads from S3, there is no easy way to >> identify them using current metrics. >> >> This KIP proposes adding client-id tags to the remote fetch metrics in >> the RemoteLogManager. This enables operators to attribute remote fetch >> costs to specific consumers for chargeback and governance purposes. >> >> Looking forward to your feedback. >> >> Regards, >> >> Viquar khan >> >> https://www.linkedin.com/in/vaquar-khan-b695577/ >> >> > > -- > Regards, > Vaquar Khan > > -- Regards, Vaquar Khan
