CRZbulabula opened a new pull request, #17714: URL: https://github.com/apache/iotdb/pull/17714
## Summary `PartiteGraphPlacementRegionGroupAllocator` ignored its `databaseAllocatedRegionGroups` argument, so PGP balanced only the global per-DataNode region count and let one DataNode hold many replicas of one database while holding few of another. On a 20-DataNode / 2-database cluster with 60 DataRegion groups per database (replication factor 3), this could leave DN-16 with 15 replicas of `tod_sod0` but only 3 of `usr_sod0`. That imbalance bled into the leader balancer: per-(database, DataNode) leader counts that should have been a flat 3 ranged from 2 to 4, with one DataNode holding 8 leaders total. This PR makes both PGP and the Greedy fallback aware of the per-database load. ## Approach Comparison priority (smaller is better) for both allocators: 1. **`regionSum`** — total per-DataNode region count (global balance) 2. **`databaseRegionSum`** — per-(database, DataNode) region count (new) 3. **`edgeSum`** — 2-region scatter (PGP only) PGP's partite-graph structure already provides the high-scatter property by construction, so demoting `edgeSum` to the tertiary key does not regress scatter width. ### PartiteGraphPlacementRegionGroupAllocator - `prepare()` now consumes `databaseAllocatedRegionGroups` and builds a `databaseRegionCounter[]` parallel to `regionCounter[]`. - `valuation()` returns a `Value(regionSum, databaseRegionSum, edgeSum)` triple; `subGraphSearch` and `partiteGraphSearch` compare via `Value.compareTo`. - The sub-graph pre-sort uses a new internal `PgpDataNodeEntry` ordered by `(regionCount, databaseRegionCount, freeDiskSpace, random)` so the fixed alpha slots also honour per-database balance. ### GreedyRegionGroupAllocator - `DataNodeEntry` gains a `databaseRegionCount` field; `compareTo` orders by `(regionCount, databaseRegionCount, freeDiskSpace, random)`. - `buildWeightList` accepts and accumulates `databaseAllocatedRegionGroups`. ### IT framework `CommonConfig` (and the `Mpp` / `Shared` / `Remote` implementations) gains `setRegionGroupAllocatePolicy(String)` so integration tests can switch between PGR / GCR / GREEDY. ## Test plan - [x] `mvn test -pl iotdb-core/confignode -Dtest='*RegionGroupAllocator*Test'` — 9 unit tests pass. - [x] New `PartiteGraphPlacementRegionGroupAllocatorTest` covers replication factor 2 / 3 / 5 multi-database scenarios. The 20-DN / 2-db / rf-3 regression now distributes each database's 60 region groups as exactly 9 replicas per DataNode (max − min = 0). - [x] New per-database balance test in `GreedyRegionGroupAllocatorTest`. - [x] `mvn verify -DskipUTs -Dit.test='IoTDBPerDatabaseRegionGroupAllocationIT#testPgrPolicyPerDbReplicaBalance' -PClusterIT -P with-integration-tests` — PGR end-to-end IT passes on a 1C4D cluster. - [x] Same IT also passes for `GCR` policy. - [x] `mvn spotless:apply` — clean. ## Out of scope / follow-ups - This PR only affects new region group allocation. Pre-existing region groups are not rebalanced. The follow-up to rebalance an existing cluster would need a separate migration tool and is not addressed here. - The leader balancer's separate-balancing-per-region-type behaviour (which also contributes to combined imbalance) is unchanged in this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
