suzhiking opened a new pull request, #23803:
URL: https://github.com/apache/kafka/pull/23803
Appends `BalancedTaskAssignor` to `GroupCoordinatorConfig`'s list of
built-in streams task assignors, after `StickyTaskAssignor`. The default of
`group.streams.assignors` becomes `sticky,balanced`, so every broker
registers the balanced assignor without configuration and a group can
select it with `streams.assignor.name=balanced`. The first entry is
unchanged, so groups that do not set `streams.assignor.name` keep using the
sticky assignor.
The assignor was left unregistered when it was added (KAFKA-20998, #23419)
because its active-task fix-up loop compared every pair of processes on each
pass and took up to 26 times the sticky assignor's time for 1000
single-member processes. #23755 replaced that loop with the Sainte-Laguë
method for active tasks. JMH, full assignment, 1000 members, 100
subtopologies of 100 partitions, ms:
| processes | standbys | balanced | sticky |
|---|---|---|---|
| 1000 × 1 member | 0 | 1.65 | 2.96 |
| 1000 × 1 member | 1 | 10.19 | 4.70 |
| 20 × 50 members | 0 | 1.74 | 2.83 |
| 20 × 50 members | 1 | 4.34 | 5.14 |
| 400 of 1–4 members | 0 | 1.68 | 3.30 |
| 400 of 1–4 members | 1 | 4.36 | 4.68 |
With standbys the remaining time is mostly the standby placement loop, which
is to be redesigned together with rack-aware standby placement
(KAFKA-20999).
Changes:
- `GroupCoordinatorConfig`: append `new BalancedTaskAssignor()` to
`STREAMS_GROUP_BUILTIN_ASSIGNORS`.
- `GroupCoordinatorConfigTest`: the default list is now
`["sticky", "balanced"]`.
- `StreamsGroupHeartbeatRequestTest.testAlterStreamsAssignorNameGroupConfig`:
a group sets `streams.assignor.name=balanced` through
`incrementalAlterConfigs` without any broker configuration, and the
described value becomes `balanced`.
- `streams-rebalance-protocol.md`: describe both built-in assignors, and
replace the "High Availability Assignor" not-yet-available entry with
rack-aware standby assignment, which neither built-in assignor supports
yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]