J-HowHuang opened a new pull request, #19221: URL: https://github.com/apache/pinot/pull/19221
## Description Currently it's `SegmentStatusChecker` in controllers reporting replica-related gauges. There are few down sides: 1. Far from where the truth land, as these gauges should reflect how the replicas status in the context of queries. For example, it doesn't account for strict replica group limitation to exclude unavailable instance for other segments in the entire replica groups 3. It's doing expensive ZK metadata access per segment 4. Duplicate implementation of "grace period for new segments" A better idea is to piggy-back on what we already have in broker's instance selector of a table's routing entry, i.e. grace period for new segments, EV & IS change listener, strict replica group handling. ## Changes Three gauges, all per-table, all instantaneous, emitted from the broker's routing table: ### `BrokerGauge.PERCENT_OF_REPLICAS` Worst segment's routable replicas as a percentage of its assigned replicas — the minimum across the table, each segment measured against its own assignment. Excluding segments with only one replica, or new segment that's fresher than `pinot.broker.new.segment.expiration.seconds`. ### `BrokerGauge.SEGMENTS_WITHOUT_REDUNDANCY` How many segments are down to their last routable replica, or have none left — the blast radius behind the percentage. Excluding segments with only one replica, or new segment that's fresher than `pinot.broker.new.segment.expiration.seconds`. ### `BrokerGauge.UNAVAILABLE_SEGMENTS` How many segments cannot be routed anywhere, i.e. what queries actually fail on with `BROKER_SEGMENT_UNAVAILABLE`. Segments with only one replica is included. Excluding new segment that's fresher than `pinot.broker.new.segment.expiration.seconds`. ## Note Since now this is a broker metric, each table would get as many time series as it's assigned brokers. Monitor the `min` over the time series if interested in the worst scenario. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
