KKcorps opened a new pull request, #20008: URL: https://github.com/apache/pinot/pull/20008
This is the backward compatible version of #19730, so the two can be compared. When a stream partition ends, like a Kinesis shard that is fully read after a split or merge, Pinot never creates another consuming segment for it, and alerts on `llcPartitionConsuming` keep firing for a partition that is done. The controller side is the same as #19730. The lead controller of the table emits a `partitionEndOfLife` gauge for each partition: 1 when a partition stops for end of life, and 0 once such a partition gets a consuming segment again. Other partitions emit nothing, and the gauge is removed when the partition has no segments left or leadership moves. Alerts can skip partitions where it is 1. This gauge is new, so nothing existing changes. The server fix is off by default. `goOnlineFromConsuming` sets `llcPartitionConsuming` back to 0 after the replace already offloaded the segment and removed the gauge, so the gauge stays at 0 forever for an ended partition. Setting `pinot.server.instance.realtime.remove.consuming.gauge.after.replace=true` skips that set, which fixes plain `== 0` alerts without the controller gauge. It stays off because some alerts may rely on that 0 to catch a partition whose next segment was never created. Tests: the controller tests from #19730, plus a server test with the setting on and off. Each new test fails without its fix. Spotless, checkstyle and license pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
