gabriellefu commented on PR #23588: URL: https://github.com/apache/kafka/pull/23588#issuecomment-5923776014
### JMH: `StreamsAssignorBenchmark`, sticky assignor, trunk vs this PR **Setup** - Base: trunk `aee48fdba1`; PR: `2c2226d79c`. Both runs use this PR's benchmark (trunk's has no `rackAwareTags` param); trunk's sticky assignor ignores the tags. - `-p assignorType=STICKY -p reportedOffsets=NONE -p standbyReplicas=1,2 -p rackAwareTags=NONE,CLUSTER_AND_ZONE`, other params at their defaults (memberCount 100/1000, membersPerProcess 1/50, partitionCount 10/100, subtopologyCount 10/100, FULL/INCREMENTAL): 128 configurations. - `CLUSTER_AND_ZONE` sets `rack.aware.assignment.tags=cluster,zone`; the processes take 2 clusters and 3 zones round-robin, so all 6 combinations occur evenly. Every second subtopology is stateful. - 5 warmup + 5 measurement iterations of 2 s, 1 fork, average time per assignment. Apple M5 Max, JDK 25. **1. Groups without `rack.aware.assignment.tags`: no regression.** PR / trunk geometric mean over the 64 configurations: 1.02x (0.95x to 1.13x; the largest differences are on configurations below 0.5 ms). **2. Groups with tags: the cost grows with the number of processes.** PR / trunk, range over FULL/INCREMENTAL, 1/2 standby replicas and the partition/subtopology split: | processes | 100 tasks | 1,000 tasks | 10,000 tasks | |---|---|---|---| | 2 | 1.5x to 1.7x | 1.6x to 2.0x | 1.5x to 1.6x | | 20 | 1.9x to 2.6x | 2.5x to 3.8x | 4.0x to 4.4x | | 100 | 4.0x to 6.3x | 5.3x to 11.0x | 4.9x to 7.6x | | 1000 | 10.5x to 24.2x | 24.3x to 53.1x | 50.6x to 90.2x | Absolute time per assignment with tags, 10,000 tasks (100 partitions x 100 subtopologies): | processes | assignment | standby replicas | trunk ms | PR ms | PR / trunk | |---|---|---|---|---|---| | 2 | FULL | 1 | 3.67 | 5.81 | 1.6x | | 2 | FULL | 2 | 3.81 | 5.96 | 1.6x | | 2 | INCREMENTAL | 1 | 3.19 | 4.75 | 1.5x | | 2 | INCREMENTAL | 2 | 3.94 | 5.75 | 1.5x | | 20 | FULL | 1 | 3.98 | 16.28 | 4.1x | | 20 | FULL | 2 | 4.96 | 21.83 | 4.4x | | 20 | INCREMENTAL | 1 | 3.16 | 13.18 | 4.2x | | 20 | INCREMENTAL | 2 | 4.40 | 17.52 | 4.0x | | 100 | FULL | 1 | 4.05 | 29.53 | 7.3x | | 100 | FULL | 2 | 5.07 | 38.36 | 7.6x | | 100 | INCREMENTAL | 1 | 3.26 | 15.90 | 4.9x | | 100 | INCREMENTAL | 2 | 4.52 | 28.50 | 6.3x | | 1000 | FULL | 1 | 4.13 | 312.72 | 75.8x | | 1000 | FULL | 2 | 4.91 | 442.66 | 90.2x | | 1000 | INCREMENTAL | 1 | 3.38 | 170.96 | 50.6x | | 1000 | INCREMENTAL | 2 | 4.45 | 302.07 | 67.8x | **3. Why.** Each rack-aware pick scans the candidate processes: all of them for the first pick of a task and again after a tag key is given up, then the survivors. For tagged groups the standby assignment is therefore O(T · (R + K) · (P + M)), against O(T · R · log P) for the tag-blind path (T stateful tasks, R standby replicas, K tag keys, P processes, M members). The measurements match: 10x tasks gives 10x the extra time and 10x processes about 12x, roughly 30 ns per (task, process) pair. FULL costs more than INCREMENTAL because without previous members every task also goes through the least-loaded rack-aware step. The partition/subtopology split does not matter, only the number of tasks. **Possible follow-up.** Group the processes by their tag values (6 groups here) and keep a heap by load per group, so that a pick looks at the groups instead of every process (about O(B · K + R · log P) per pick, B = distinct tag value combinations). This can be a follow-up if the current cost is a concern. <details><summary>All 128 configurations</summary> | assignment | members | members per process | partitions | subtopologies | standby replicas | tags | trunk ms | PR ms | PR / trunk | |---|---|---|---|---|---|---|---|---|---| | FULL | 100 | 1 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.045 | 0.253 | 5.66x | | FULL | 100 | 1 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.049 | 0.311 | 6.32x | | FULL | 100 | 1 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.290 | 2.294 | 7.91x | | FULL | 100 | 1 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.334 | 3.519 | 10.55x | | FULL | 100 | 1 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.267 | 2.367 | 8.88x | | FULL | 100 | 1 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.313 | 3.441 | 10.98x | | FULL | 100 | 1 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 4.053 | 29.529 | 7.29x | | FULL | 100 | 1 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 5.074 | 38.356 | 7.56x | | FULL | 100 | 50 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.034 | 0.058 | 1.71x | | FULL | 100 | 50 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.037 | 0.064 | 1.72x | | FULL | 100 | 50 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.279 | 0.469 | 1.68x | | FULL | 100 | 50 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.300 | 0.588 | 1.96x | | FULL | 100 | 50 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.256 | 0.445 | 1.74x | | FULL | 100 | 50 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.276 | 0.501 | 1.81x | | FULL | 100 | 50 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 3.670 | 5.806 | 1.58x | | FULL | 100 | 50 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 3.807 | 5.957 | 1.56x | | FULL | 1000 | 1 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.144 | 3.043 | 21.08x | | FULL | 1000 | 1 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.153 | 3.710 | 24.21x | | FULL | 1000 | 1 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.591 | 26.420 | 44.68x | | FULL | 1000 | 1 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.676 | 32.619 | 48.23x | | FULL | 1000 | 1 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.565 | 25.975 | 45.93x | | FULL | 1000 | 1 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.645 | 34.245 | 53.06x | | FULL | 1000 | 1 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 4.127 | 312.722 | 75.77x | | FULL | 1000 | 1 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 4.906 | 442.658 | 90.23x | | FULL | 1000 | 50 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.093 | 0.214 | 2.30x | | FULL | 1000 | 50 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.102 | 0.262 | 2.57x | | FULL | 1000 | 50 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.514 | 1.636 | 3.18x | | FULL | 1000 | 50 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.624 | 2.238 | 3.59x | | FULL | 1000 | 50 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.477 | 1.586 | 3.32x | | FULL | 1000 | 50 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.571 | 2.196 | 3.85x | | FULL | 1000 | 50 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 3.980 | 16.283 | 4.09x | | FULL | 1000 | 50 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 4.958 | 21.832 | 4.40x | | INCREMENTAL | 100 | 1 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.042 | 0.168 | 4.02x | | INCREMENTAL | 100 | 1 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.048 | 0.232 | 4.81x | | INCREMENTAL | 100 | 1 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.267 | 1.414 | 5.30x | | INCREMENTAL | 100 | 1 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.338 | 2.598 | 7.70x | | INCREMENTAL | 100 | 1 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.206 | 1.366 | 6.64x | | INCREMENTAL | 100 | 1 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.254 | 2.516 | 9.91x | | INCREMENTAL | 100 | 1 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 3.261 | 15.899 | 4.88x | | INCREMENTAL | 100 | 1 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 4.525 | 28.500 | 6.30x | | INCREMENTAL | 100 | 50 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.032 | 0.047 | 1.48x | | INCREMENTAL | 100 | 50 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.039 | 0.057 | 1.49x | | INCREMENTAL | 100 | 50 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.254 | 0.403 | 1.58x | | INCREMENTAL | 100 | 50 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.321 | 0.499 | 1.56x | | INCREMENTAL | 100 | 50 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.205 | 0.354 | 1.73x | | INCREMENTAL | 100 | 50 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.271 | 0.445 | 1.64x | | INCREMENTAL | 100 | 50 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 3.195 | 4.746 | 1.49x | | INCREMENTAL | 100 | 50 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 3.935 | 5.746 | 1.46x | | INCREMENTAL | 1000 | 1 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.151 | 1.590 | 10.53x | | INCREMENTAL | 1000 | 1 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.158 | 2.511 | 15.89x | | INCREMENTAL | 1000 | 1 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.566 | 13.732 | 24.28x | | INCREMENTAL | 1000 | 1 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.696 | 21.255 | 30.53x | | INCREMENTAL | 1000 | 1 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.501 | 13.511 | 26.96x | | INCREMENTAL | 1000 | 1 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.590 | 20.469 | 34.68x | | INCREMENTAL | 1000 | 1 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 3.378 | 170.956 | 50.61x | | INCREMENTAL | 1000 | 1 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 4.454 | 302.074 | 67.82x | | INCREMENTAL | 1000 | 50 | 10 | 10 | 1 | CLUSTER_AND_ZONE | 0.083 | 0.154 | 1.85x | | INCREMENTAL | 1000 | 50 | 10 | 10 | 2 | CLUSTER_AND_ZONE | 0.092 | 0.204 | 2.22x | | INCREMENTAL | 1000 | 50 | 10 | 100 | 1 | CLUSTER_AND_ZONE | 0.432 | 1.094 | 2.53x | | INCREMENTAL | 1000 | 50 | 10 | 100 | 2 | CLUSTER_AND_ZONE | 0.562 | 1.735 | 3.09x | | INCREMENTAL | 1000 | 50 | 100 | 10 | 1 | CLUSTER_AND_ZONE | 0.379 | 1.032 | 2.72x | | INCREMENTAL | 1000 | 50 | 100 | 10 | 2 | CLUSTER_AND_ZONE | 0.500 | 1.637 | 3.27x | | INCREMENTAL | 1000 | 50 | 100 | 100 | 1 | CLUSTER_AND_ZONE | 3.163 | 13.183 | 4.17x | | INCREMENTAL | 1000 | 50 | 100 | 100 | 2 | CLUSTER_AND_ZONE | 4.396 | 17.518 | 3.99x | | FULL | 100 | 1 | 10 | 10 | 1 | NONE | 0.045 | 0.043 | 0.96x | | FULL | 100 | 1 | 10 | 10 | 2 | NONE | 0.051 | 0.051 | 0.99x | | FULL | 100 | 1 | 10 | 100 | 1 | NONE | 0.290 | 0.305 | 1.05x | | FULL | 100 | 1 | 10 | 100 | 2 | NONE | 0.348 | 0.332 | 0.95x | | FULL | 100 | 1 | 100 | 10 | 1 | NONE | 0.278 | 0.269 | 0.97x | | FULL | 100 | 1 | 100 | 10 | 2 | NONE | 0.300 | 0.302 | 1.01x | | FULL | 100 | 1 | 100 | 100 | 1 | NONE | 3.942 | 3.910 | 0.99x | | FULL | 100 | 1 | 100 | 100 | 2 | NONE | 4.943 | 5.107 | 1.03x | | FULL | 100 | 50 | 10 | 10 | 1 | NONE | 0.034 | 0.036 | 1.05x | | FULL | 100 | 50 | 10 | 10 | 2 | NONE | 0.037 | 0.038 | 1.03x | | FULL | 100 | 50 | 10 | 100 | 1 | NONE | 0.291 | 0.282 | 0.97x | | FULL | 100 | 50 | 10 | 100 | 2 | NONE | 0.312 | 0.304 | 0.98x | | FULL | 100 | 50 | 100 | 10 | 1 | NONE | 0.254 | 0.259 | 1.02x | | FULL | 100 | 50 | 100 | 10 | 2 | NONE | 0.289 | 0.287 | 0.99x | | FULL | 100 | 50 | 100 | 100 | 1 | NONE | 3.507 | 3.475 | 0.99x | | FULL | 100 | 50 | 100 | 100 | 2 | NONE | 3.520 | 3.962 | 1.13x | | FULL | 1000 | 1 | 10 | 10 | 1 | NONE | 0.144 | 0.155 | 1.08x | | FULL | 1000 | 1 | 10 | 10 | 2 | NONE | 0.151 | 0.161 | 1.06x | | FULL | 1000 | 1 | 10 | 100 | 1 | NONE | 0.585 | 0.594 | 1.02x | | FULL | 1000 | 1 | 10 | 100 | 2 | NONE | 0.689 | 0.710 | 1.03x | | FULL | 1000 | 1 | 100 | 10 | 1 | NONE | 0.544 | 0.554 | 1.02x | | FULL | 1000 | 1 | 100 | 10 | 2 | NONE | 0.647 | 0.670 | 1.04x | | FULL | 1000 | 1 | 100 | 100 | 1 | NONE | 3.938 | 4.068 | 1.03x | | FULL | 1000 | 1 | 100 | 100 | 2 | NONE | 4.939 | 4.825 | 0.98x | | FULL | 1000 | 50 | 10 | 10 | 1 | NONE | 0.097 | 0.100 | 1.03x | | FULL | 1000 | 50 | 10 | 10 | 2 | NONE | 0.101 | 0.107 | 1.06x | | FULL | 1000 | 50 | 10 | 100 | 1 | NONE | 0.513 | 0.513 | 1.00x | | FULL | 1000 | 50 | 10 | 100 | 2 | NONE | 0.620 | 0.637 | 1.03x | | FULL | 1000 | 50 | 100 | 10 | 1 | NONE | 0.477 | 0.486 | 1.02x | | FULL | 1000 | 50 | 100 | 10 | 2 | NONE | 0.601 | 0.590 | 0.98x | | FULL | 1000 | 50 | 100 | 100 | 1 | NONE | 3.987 | 4.041 | 1.01x | | FULL | 1000 | 50 | 100 | 100 | 2 | NONE | 5.100 | 5.020 | 0.98x | | INCREMENTAL | 100 | 1 | 10 | 10 | 1 | NONE | 0.042 | 0.043 | 1.02x | | INCREMENTAL | 100 | 1 | 10 | 10 | 2 | NONE | 0.048 | 0.051 | 1.06x | | INCREMENTAL | 100 | 1 | 10 | 100 | 1 | NONE | 0.265 | 0.271 | 1.02x | | INCREMENTAL | 100 | 1 | 10 | 100 | 2 | NONE | 0.327 | 0.335 | 1.03x | | INCREMENTAL | 100 | 1 | 100 | 10 | 1 | NONE | 0.208 | 0.210 | 1.01x | | INCREMENTAL | 100 | 1 | 100 | 10 | 2 | NONE | 0.259 | 0.259 | 1.00x | | INCREMENTAL | 100 | 1 | 100 | 100 | 1 | NONE | 3.361 | 3.331 | 0.99x | | INCREMENTAL | 100 | 1 | 100 | 100 | 2 | NONE | 4.471 | 4.534 | 1.01x | | INCREMENTAL | 100 | 50 | 10 | 10 | 1 | NONE | 0.032 | 0.031 | 0.99x | | INCREMENTAL | 100 | 50 | 10 | 10 | 2 | NONE | 0.039 | 0.040 | 1.04x | | INCREMENTAL | 100 | 50 | 10 | 100 | 1 | NONE | 0.254 | 0.267 | 1.05x | | INCREMENTAL | 100 | 50 | 10 | 100 | 2 | NONE | 0.321 | 0.338 | 1.05x | | INCREMENTAL | 100 | 50 | 100 | 10 | 1 | NONE | 0.208 | 0.213 | 1.03x | | INCREMENTAL | 100 | 50 | 100 | 10 | 2 | NONE | 0.267 | 0.282 | 1.06x | | INCREMENTAL | 100 | 50 | 100 | 100 | 1 | NONE | 3.220 | 3.245 | 1.01x | | INCREMENTAL | 100 | 50 | 100 | 100 | 2 | NONE | 4.114 | 4.059 | 0.99x | | INCREMENTAL | 1000 | 1 | 10 | 10 | 1 | NONE | 0.148 | 0.156 | 1.06x | | INCREMENTAL | 1000 | 1 | 10 | 10 | 2 | NONE | 0.156 | 0.163 | 1.05x | | INCREMENTAL | 1000 | 1 | 10 | 100 | 1 | NONE | 0.548 | 0.560 | 1.02x | | INCREMENTAL | 1000 | 1 | 10 | 100 | 2 | NONE | 0.683 | 0.687 | 1.01x | | INCREMENTAL | 1000 | 1 | 100 | 10 | 1 | NONE | 0.479 | 0.489 | 1.02x | | INCREMENTAL | 1000 | 1 | 100 | 10 | 2 | NONE | 0.616 | 0.623 | 1.01x | | INCREMENTAL | 1000 | 1 | 100 | 100 | 1 | NONE | 3.532 | 3.501 | 0.99x | | INCREMENTAL | 1000 | 1 | 100 | 100 | 2 | NONE | 4.526 | 4.613 | 1.02x | | INCREMENTAL | 1000 | 50 | 10 | 10 | 1 | NONE | 0.083 | 0.089 | 1.07x | | INCREMENTAL | 1000 | 50 | 10 | 10 | 2 | NONE | 0.090 | 0.101 | 1.12x | | INCREMENTAL | 1000 | 50 | 10 | 100 | 1 | NONE | 0.428 | 0.441 | 1.03x | | INCREMENTAL | 1000 | 50 | 10 | 100 | 2 | NONE | 0.556 | 0.573 | 1.03x | | INCREMENTAL | 1000 | 50 | 100 | 10 | 1 | NONE | 0.371 | 0.382 | 1.03x | | INCREMENTAL | 1000 | 50 | 100 | 10 | 2 | NONE | 0.489 | 0.525 | 1.07x | | INCREMENTAL | 1000 | 50 | 100 | 100 | 1 | NONE | 3.186 | 3.286 | 1.03x | | INCREMENTAL | 1000 | 50 | 100 | 100 | 2 | NONE | 4.403 | 4.546 | 1.03x | </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
