Hi Alieh, I'm so sorry for the delay in replying. Thank you for the thoughtful questions and suggestions. They were very helpful in clarifying the scope and semantics of the proposed metric.
I have updated the KIP accordingly. Please find my responses below. AS1. Thank you for raising this! I agree that a cumulative counter could help preserve statuses that appear and clear between metric reads. My current inclination is to keep the gauge without adding a counter. The main goal of this KIP is to help operators diagnose a problem that is currently affecting a Kafka Streams client, based on its latest successful heartbeat response, rather than to retain a history of statuses observed between metric collection intervals. A response-based counter would also depend on the configured heartbeat interval and would count heartbeat responses rather than distinct incidents. A transition-based counter could avoid that dependency, but it would require additional state and transition semantics. Given this scope, I think the gauge may be sufficient, with the explicit trade-off that a status appearing and clearing entirely between metric reads may not be observed. I have added this trade-off to the Rejected Alternatives section of the KIP. Would keeping the gauge with this documented limitation seem reasonable to you? I would be happy to reconsider the counter if preserving transient status history is considered an important requirement for this KIP! AS2. Yes, heartbeat response processing and metric reading may occur on different threads. This communication is already handled by StreamsRebalanceData, which stores the latest heartbeat status collection in an AtomicReference. The consumer background thread publishes the collection, while the gauge may be evaluated by another thread, such as a JMX request-handling thread, a MetricsReporter thread, or an application thread reading Kafka Streams metrics. AtomicReference provides atomic replacement and the required memory visibility between the writer and readers. A read performed after an update has completed observes the latest published collection. If a read is concurrent with an update, it may observe either the previous or the new collection, but not a partially published reference. Individual metric values are read independently, as with other Kafka metrics, so a collection that spans an update is not guaranteed to be a transactional snapshot across all status metrics. I added a thread-safety section to clarify this behavior. AS3. I agree that streams-group-status may imply that the metric represents the state of the entire Streams group, while it actually represents the statuses observed by an individual stream thread. I have renamed the metric to heartbeat-status. Its thread-level scope is identified by the stream-thread-metrics group and the thread-id tag. AS4. I agree that the edge-case behavior should be explicitly documented. I added the following semantics to the KIP: - Before the first successful heartbeat response, all status gauges return 0. - For a successful response containing one or more statuses, gauges for the contained statuses return 1 and all other status gauges return 0. - For a successful response containing no statuses, all status gauges return 0. - A heartbeat response with an error, or a request that fails, does not change the gauge values. The previous values are retained. If no successful response has been received, all gauges remain 0. A failed heartbeat does not provide a new authoritative status snapshot, so it neither clears nor activates any status. The updated KIP is available here: https://cwiki.apache.org/confluence/spaces/KAFKA/pages/429064565/KIP-1350+Expose+Streams+Group+Heartbeat+Statuses+as+Kafka+Streams+Metrics Thank you again for reviewing the KIP! I would be happy to address any additional questions or suggestions. Best regards, Sanghyeok An 2026년 6월 11일 (목) 오전 1:10, Alieh Saeedi via dev <[email protected]>님이 작성: > Hey Sanghyeok An > > Thanks for the KIP. A couple of questions/considerations: > > AS1- A gauge only shows the latest heartbeat. Could we discuss a cumulative > counter or the gauge vs counter trade-off in the KIP? I don't think the > gauge is wrong, but on its own it's hard to alert on reliably. > > AS2. Are the metric reader and writer threads the same? I assume they are > two different threads. Could we discuss on the race conditions? Is it > guaranteed that the reader sees the latest write? > > AS3- The name `streams-group-status` reads like a property of the whole > group, but it's a per-thread metric (thread-id tag) reporting that thread's > last heartbeat. Would something like `heartbeat-status` work better? > > AS4- We may need to list more edge cases?! For example, when a heartbeat > has no status or when it fails. I mean could we list and discuss them? What > should the value be (0 or 1) in each case? > > Thanks, > Alieh > > On Wed, Jun 10, 2026 at 12:30 AM Sanghyeok An <[email protected]> wrote: > > > Hello Kafka Community, > > > > I would like to gently follow up on the discussion for KIP-1350, in case > > anyone has had a chance to take a look. > > This KIP proposes exposing Streams group heartbeat statuses as Kafka > > Streams metrics, to make it easier to diagnose Streams rebalance protocol > > issues without relying only on client logs. > > > > KIP - > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1350*3A*Expose*Streams*Group*Heartbeat*Statuses*as*Kafka*Streams*Metrics__;JSsrKysrKysrKw!!Ayb5sqE7!uEAEaq5_dYw5iEExhfHl_jW2SEqSm3y_J7B-aBKm_otsCrnovllLW3t0MVpDt30NO5j73-tCwHgeOtxo6EM$ > > > > I would greatly appreciate any feedback, comments, or suggestions you may > > have. > > > > Best regards, > > Sanghyeok An. > > > > 2026년 6월 1일 (월) 오후 10:47, Sanghyeok An <[email protected]>님이 작성: > > > > > Hello Kafka Community, > > > > > > I would like to gently follow up on the discussion for KIP-1350, in > case > > > anyone has had a chance to take a look. > > > This KIP proposes exposing Streams group heartbeat statuses as Kafka > > > Streams metrics, to make it easier to diagnose Streams rebalance > protocol > > > issues without relying only on client logs. > > > > > > I would greatly appreciate any feedback, comments, or suggestions you > may > > > have. > > > > > > Best regards, > > > Sanghyeok An. > > > > > >
