gabriellefu commented on PR #23588: URL: https://github.com/apache/kafka/pull/23588#issuecomment-5982459351
after update the algorithm, the jhm benchmark's result is now as follows: ### JMH (updated): `StreamsAssignorBenchmark`, sticky assignor, trunk vs this PR Rerun on `a238583b12` after the review changes. This replaces the numbers in my earlier JMH comment, which were measured on `2c2226d79c`. **Setup** - Base: trunk `aee48fdba1`; PR: `a238583b12`. Both runs use this PR's benchmark (trunk's has no `rackAwareTags` param); trunk's sticky assignor ignores the tags. - `-p assignorType=STICKY -p reportedOffsets=NONE -p standbyReplicas=1,2 -p rackAwareTags=NONE,CLUSTER_AND_ZONE`, other params at their defaults (memberCount 100/1000, membersPerProcess 1/50, partitionCount 10/100, subtopologyCount 10/100, FULL/INCREMENTAL): 128 configurations. - `CLUSTER_AND_ZONE` sets `rack.aware.assignment.tags=cluster,zone`; the processes take 2 clusters and 3 zones round-robin, so all 6 combinations occur evenly. Every second subtopology is stateful. - 5 warmup + 5 measurement iterations of 2 s, 1 fork, average time per assignment. Apple M5 Max, JDK 25. **1. Groups without `rack.aware.assignment.tags`: no regression.** PR / trunk geometric mean over the 64 configurations: 1.05x (0.97x to 1.15x; the largest differences are on configurations below 0.5 ms). **2. Groups with tags: the cost no longer grows with the number of processes.** PR / trunk, range over FULL/INCREMENTAL, 1/2 standby replicas and the partition/subtopology split: | processes | 100 tasks | 1,000 tasks | 10,000 tasks | |---|---|---|---| | 2 | 1.2x to 1.3x | 1.2x to 1.5x | 1.2x to 1.4x | | 20 | 1.2x to 1.4x | 1.3x to 1.5x | 1.4x to 1.6x | | 100 | 1.5x to 1.6x | 1.5x to 2.1x | 1.4x to 1.5x | | 1000 | 1.6x | 1.4x to 1.7x | 1.4x to 1.7x | Geometric mean over the 64 configurations: 1.44x. On `2c2226d79c`, 1000 processes cost 10.5x to 90.2x. Absolute time per assignment with tags, 10,000 tasks (100 partitions x 100 subtopologies): | processes | assignment | standby replicas | trunk ms | PR ms | PR / trunk | |---|---|---|---|---|---| | 2 | FULL | 1 | 3.59 | 4.27 | 1.2x | | 2 | FULL | 2 | 3.60 | 4.68 | 1.3x | | 2 | INCREMENTAL | 1 | 3.16 | 4.16 | 1.3x | | 2 | INCREMENTAL | 2 | 3.88 | 5.27 | 1.4x | | 20 | FULL | 1 | 3.96 | 5.44 | 1.4x | | 20 | FULL | 2 | 4.89 | 6.90 | 1.4x | | 20 | INCREMENTAL | 1 | 3.29 | 4.65 | 1.4x | | 20 | INCREMENTAL | 2 | 4.32 | 7.04 | 1.6x | | 100 | FULL | 1 | 3.87 | 5.37 | 1.4x | | 100 | FULL | 2 | 5.01 | 6.99 | 1.4x | | 100 | INCREMENTAL | 1 | 3.16 | 4.51 | 1.4x | | 100 | INCREMENTAL | 2 | 4.45 | 6.58 | 1.5x | | 1000 | FULL | 1 | 3.96 | 5.38 | 1.4x | | 1000 | FULL | 2 | 4.92 | 7.97 | 1.6x | | 1000 | INCREMENTAL | 1 | 3.50 | 5.13 | 1.5x | | 1000 | INCREMENTAL | 2 | 4.51 | 7.52 | 1.7x | Every configuration stays under 8 ms; the slowest is 1000 processes, FULL, 2 standby replicas: 7.97 ms against 4.92 ms on trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
