gabriellefu commented on PR #23588:
URL: https://github.com/apache/kafka/pull/23588#issuecomment-6093495624
### JMH results
Run with `StreamsAssignorBenchmark` on 31c3627126 (sticky assignor): 1000
members, 100 subtopologies with 100 partitions each (10,000 tasks), no reported
offsets. A cell is the average time of one assignment in ms (1 fork, 5 × 5s
warmup, 5 × 5s measurement). **Full** is the assignment from empty;
**incremental** is the assignment after one member joins. Each process takes
the value of every tag key round-robin, and `host` has a value of its own per
process, so a tag group is a set of processes with the same values for
`rack.aware.assignment.tags`.
**1000 processes (1 member per process)**
| rack.aware.assignment.tags | tag groups | full, 1 standby | full, 2
standbys | incremental, 1 standby | incremental, 2 standbys |
|---|---|---|---|---|---|
| none | n/a | 4.19 | 5.00 | 3.50 | 4.70 |
| cluster, zone | 6 | 6.07 | 8.40 | 5.91 | 8.58 |
| cluster, zone, rack | 42 | 6.91 | 10.38 | 6.30 | 10.02 |
| cluster, zone, host | 1000 | 6.81 | 9.91 | 6.72 | 10.19 |
| region, cluster, zone, rack, host | 1000 | 9.08 | 15.28 | 7.59 | 12.20 |
**20 processes (50 members per process)**
| rack.aware.assignment.tags | tag groups | full, 1 standby | full, 2
standbys | incremental, 1 standby | incremental, 2 standbys |
|---|---|---|---|---|---|
| none | n/a | 4.50 | 5.46 | 3.32 | 4.55 |
| cluster, zone | 6 | 5.68 | 8.05 | 5.00 | 8.15 |
| cluster, zone, rack | 20 | 6.30 | 10.06 | 5.63 | 9.86 ± 2.62 |
| cluster, zone, host | 20 | 6.06 | 9.25 | 5.39 | 8.58 |
| region, cluster, zone, rack, host | 20 | 7.48 | 12.10 | 6.45 | 10.57 |
Without rack-aware assignment tags an assignment takes 3.3–5.5 ms. With two
or three tag keys it takes 5.0–10.4 ms, and with five keys 6.4–15.3 ms.
The cost does not grow with the number of tag groups: with 1000 processes,
42 and 1000 tag groups take about the same time (6.91 vs 6.81 ms with 1
standby, 10.38 vs 9.91 ms with 2 standbys). It grows with the number of tag
keys and standby replicas. The worst case is five keys, 1000 tag groups, 2
standbys and a full assignment: 15.3 ms, 10.3 ms over the tag-blind run.
The incremental run with 20 processes, three keys and 2 standbys is noisy (±
2.62 ms); every other error is within 4% of the score.
<details>
<summary>All results</summary>
```
./jmh-benchmarks/jmh.sh StreamsAssignorBenchmark \
-p assignorType=STICKY \
-p memberCount=1000 \
-p partitionCount=100 \
-p subtopologyCount=100 \
-p standbyReplicas=1,2 \
-p reportedOffsets=NONE \
-w 5s -r 5s
```
```
Benchmark (assignmentType) (assignorType)
(memberCount) (membersPerProcess) (partitionCount)
(rackAwareTags) (reportedOffsets) (standbyReplicas) (subtopologyCount) Mode
Cnt Score Error Units
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
NONE NONE 1 100 avgt 5
4.191 ± 0.126 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
NONE NONE 2 100 avgt 5
4.997 ± 0.103 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
CLUSTER_AND_ZONE NONE 1 100
avgt 5 6.070 ± 0.103 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
CLUSTER_AND_ZONE NONE 2 100
avgt 5 8.396 ± 0.174 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
CLUSTER_ZONE_AND_RACK NONE 1 100
avgt 5 6.906 ± 0.138 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
CLUSTER_ZONE_AND_RACK NONE 2 100
avgt 5 10.377 ± 0.243 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
CLUSTER_ZONE_AND_HOST NONE 1 100
avgt 5 6.811 ± 0.162 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
CLUSTER_ZONE_AND_HOST NONE 2 100
avgt 5 9.910 ± 0.163 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 1
100 avgt 5 9.078 ± 0.201 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 1 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 2
100 avgt 5 15.275 ± 0.404 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
NONE NONE 1 100 avgt 5
4.501 ± 0.053 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
NONE NONE 2 100 avgt 5
5.458 ± 0.081 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
CLUSTER_AND_ZONE NONE 1 100
avgt 5 5.680 ± 0.088 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
CLUSTER_AND_ZONE NONE 2 100
avgt 5 8.047 ± 0.112 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
CLUSTER_ZONE_AND_RACK NONE 1 100
avgt 5 6.297 ± 0.084 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
CLUSTER_ZONE_AND_RACK NONE 2 100
avgt 5 10.059 ± 0.181 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
CLUSTER_ZONE_AND_HOST NONE 1 100
avgt 5 6.057 ± 0.211 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
CLUSTER_ZONE_AND_HOST NONE 2 100
avgt 5 9.245 ± 0.115 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 1
100 avgt 5 7.480 ± 0.141 ms/op
StreamsAssignorBenchmark.doAssignment FULL STICKY
1000 50 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 2
100 avgt 5 12.104 ± 0.225 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
NONE NONE 1 100 avgt 5
3.503 ± 0.104 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
NONE NONE 2 100 avgt 5
4.696 ± 0.098 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
CLUSTER_AND_ZONE NONE 1 100
avgt 5 5.905 ± 0.129 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
CLUSTER_AND_ZONE NONE 2 100
avgt 5 8.579 ± 0.171 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
CLUSTER_ZONE_AND_RACK NONE 1 100
avgt 5 6.297 ± 0.167 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
CLUSTER_ZONE_AND_RACK NONE 2 100
avgt 5 10.016 ± 0.254 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
CLUSTER_ZONE_AND_HOST NONE 1 100
avgt 5 6.721 ± 0.221 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
CLUSTER_ZONE_AND_HOST NONE 2 100
avgt 5 10.187 ± 0.291 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 1
100 avgt 5 7.588 ± 0.259 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 1 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 2
100 avgt 5 12.199 ± 0.390 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
NONE NONE 1 100 avgt 5
3.317 ± 0.111 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
NONE NONE 2 100 avgt 5
4.548 ± 0.112 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
CLUSTER_AND_ZONE NONE 1 100
avgt 5 4.999 ± 0.150 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
CLUSTER_AND_ZONE NONE 2 100
avgt 5 8.150 ± 0.319 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
CLUSTER_ZONE_AND_RACK NONE 1 100
avgt 5 5.625 ± 0.144 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
CLUSTER_ZONE_AND_RACK NONE 2 100
avgt 5 9.864 ± 2.616 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
CLUSTER_ZONE_AND_HOST NONE 1 100
avgt 5 5.387 ± 0.134 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
CLUSTER_ZONE_AND_HOST NONE 2 100
avgt 5 8.580 ± 0.123 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 1
100 avgt 5 6.448 ± 0.159 ms/op
StreamsAssignorBenchmark.doAssignment INCREMENTAL STICKY
1000 50 100
REGION_CLUSTER_ZONE_RACK_AND_HOST NONE 2
100 avgt 5 10.571 ± 0.238 ms/op
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
