Jackie-Jiang opened a new pull request, #19916: URL: https://github.com/apache/pinot/pull/19916
## Summary Strict replica-group routing requires all segments of a partition to be hosted by the same instances. Today this is only enforced for upsert and dedup real-time tables, through `BaseStrictRealtimeSegmentAssignment`, which on every new segment scans the IdealState for an existing segment of the partition (reading segment ZK metadata for uploaded segments, and the latest segment's tier for dedup tables), and `TableRebalancer` reads partition ids from segment ZK metadata again to batch segments per partition. Offline tables with strict replica-group routing get no such guarantee. This PR records the **partition assignment** in the list fields of the table IdealState: one list field per segment partition id, holding the sorted instances hosting all segments of that partition. It is maintained for tables with strict replica-group routing (which upsert and dedup tables use): - Real-time tables: the entries describe the CONSUMING placement. CONSUMING segments, and COMPLETED segments not relocated to COMPLETED instance partitions or a tier, follow them. - Offline tables with a partition column whose OFFLINE instance partitions hold one instance per partition within each replica-group: non-tier segments follow them. The entries are written in the same IdealState update as the map fields, so they are never out of sync with the segment assignment. Helix ignores IdealState list fields in CUSTOMIZED mode. ### Segment assignment - A new segment is assigned to the entry of its partition in O(1), without scanning the IdealState or reading other segments' ZK metadata. The partition id comes from the segment name, or from the segment's own ZK metadata for offline and custom-named uploaded segments (the same single read the replica-group strategy does today). - When the partition has no entry, the entry is created from the existing segments of the partition (real-time tables created before this change, see below), else from the instance partitions. The existing mismatch meter still fires when an entry differs from what the instance partitions give. - `SegmentAssignment.assignSegment` takes the `IdealState` instead of its map fields so it can read and write the entries. `rebalanceTable` takes the `IdealState` and returns a `TableAssignment` (segment assignment + partition assignment). - `BaseStrictRealtimeSegmentAssignment`, `SingleTierStrictRealtimeSegmentAssignment` and `MultiTierStrictRealtimeSegmentAssignment` are removed: `RealtimeSegmentAssignment` and `OfflineSegmentAssignment` pin the segments through shared helpers in `BaseSegmentAssignment`. The multi-tier special case of checking the tier of the latest segment is gone, because the entry is the CONSUMING placement regardless of where old segments went. ### Rebalance - `rebalanceTable` computes the target instances of every partition from the instance partitions, and the segments follow their partition. Offline tables resolve the partition ids of all segments with a single bulk ZK read per rebalance instead of one read per segment. - The rebalance loop moves the entries along with the segments at every IdealState write: a partition takes the same step as the segments with the same current and target instances, so a segment added mid-rebalance (assigned to the entry) lands on the same instances as its siblings and is then moved with them. In strict replica-group mode the batching unit is the group of segments with the same current and target instances, sub-grouped by the partition id in the segment names; a group with a segment lacking a partition id in its name moves as a whole. This removes `PartitionIdFetcher` and its ZK reads. - Downtime rebalance writes the target entries with the target assignment. Convergence checks the entries as well as the segments. - Tables created before this change have no entries. Their first rebalance populates them: alone when no segment moves, or once the segments reach the target otherwise. Meanwhile a new real-time segment of a partition without entry is placed by the existing segments of the partition, as today. ### Behavior changes - Offline tables with strict replica-group routing, a partition column and one instance per partition within each replica-group now keep a new segment with the other segments of its partition when the instance partitions have changed, instead of placing it by the new instance partitions. - Real-time tables with strict replica-group routing that are not upsert or dedup tables get the same pinned assignment. - Upsert and dedup tables configured with the beta `multiStageReplicaGroup` routing are no longer pinned: pinning is decided by the routing config alone. Their validation is unchanged. - An old controller ignores the entries and does not update them on rebalance, so controllers should be upgraded before rebalancing; the first rebalance by a new controller rewrites all entries. The broker still ignores the list fields; using them in `SegmentPartitionMetadataManager` is a follow-up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
