gabriellefu commented on PR #23588: URL: https://github.com/apache/kafka/pull/23588#issuecomment-5959956357
### Testbed results Run with the randomized testbed from #23556 (`StickyTaskAssignorFuzzTest`, default base seed 42) on this PR at 23212ee913, against trunk at aee48fdba1. SMALL has 300 scenarios, LARGE has 30. A cell is `trunk → this PR`, averaged over the scenarios, and a single value means unchanged. **Initial** is the assignment from empty; **rebalance** is the assignment after a random change (1357 SMALL, 135 LARGE), graded against the assignment before the change. LARGE scenarios use only the `region` and `zone` keys. **Rack awareness** | metric | SMALL initial | SMALL rebalance | LARGE initial | LARGE rebalance | |---|---|---|---|---| | distinct tag values [region] (achieved / achievable) | 0.627 → 0.789 | 0.620 → 0.771 | 0.555 → 0.783 | 0.520 → 0.758 | | distinct tag values [zone] (achieved / achievable) | 0.579 → 0.709 | 0.562 → 0.697 | 0.578 → 0.790 | 0.549 → 0.718 | | distinct tag values [rack] (achieved / achievable) | 0.559 → 0.665 | 0.590 → 0.717 | n/a | n/a | | ideal distribution through [region] | 0.546 → 0.737 | 0.520 → 0.698 | 0.515 → 0.743 | 0.465 → 0.706 | | ideal distribution through [zone] | 0.311 → 0.480 | 0.257 → 0.428 | 0.329 → 0.619 | 0.294 → 0.554 | | ideal distribution through [rack] | 0.126 → 0.266 | 0.134 → 0.295 | n/a | n/a | | all replicas on one tag value [region] | 0.301 → 0.174 | 0.294 → 0.177 | 0.410 → 0.193 | 0.431 → 0.210 | | all replicas on one tag value [zone] | 0.348 → 0.229 | 0.348 → 0.221 | 0.396 → 0.192 | 0.414 → 0.253 | | all replicas on one tag value [rack] | 0.339 → 0.227 | 0.317 → 0.200 | n/a | n/a | **Stickiness** | metric | SMALL rebalance | LARGE rebalance | |---|---|---| | standby tasks moved | 0.418 → 0.599 | 9.607 → 20.133 | | standby tasks kept on process (1.0 = all) | 0.921 → 0.889 | 0.961 → 0.942 | | standby tasks kept on member (1.0 = all) | 0.912 → 0.873 | 0.957 → 0.931 | | replicas placed without state | 1.419 → 1.604 | 17.889 → 28.422 | | stateful active tasks moved | 0.470 → 0.466 | 4.548 | | stateful active tasks kept on process (1.0 = all) | 0.953 → 0.954 | 0.991 → 0.992 | The extra standby moves are the expected cost of rack diversity ranking above standby stickiness. Active tasks are not affected. **Balance** | metric | SMALL initial | SMALL rebalance | LARGE initial | LARGE rebalance | |---|---|---|---|---| | total tasks per member (max-min) | 1.080 → 1.130 | 1.419 → 1.453 | 1.233 → 1.733 | 1.467 → 1.585 | | process load (max-min) | 0.651 → 0.715 | 1.042 → 1.081 | 0.774 → 1.282 | 0.745 → 1.048 | | standby load per process (max-min) | 0.472 → 0.533 | 0.824 → 0.881 | 0.677 → 1.168 | 0.652 → 0.981 | | active tasks per member (max-min) | 0.870 | 0.855 → 0.856 | 1.000 | 1.185 → 1.200 | **Convergence** | metric | SMALL initial | SMALL rebalance | LARGE initial | LARGE rebalance | |---|---|---|---|---| | rounds to converge | 1.020 → 1.030 | 1.024 → 1.021 | 1.200 → 1.517 | 1.496 → 1.632 | | not converged | 0.000 | 0.000 | 0.000 → 0.033 | 0.000 → 0.015 | 1 of 30 LARGE initial assignments and 2 of 135 LARGE rebalances do not converge, against none on trunk. **Without rack-aware assignment tags** On the initial assignment, the tag-blind rows are identical to trunk. On a rebalance they differ only because the assignment before the change comes from the rack-aware run. <details> <summary>All metrics</summary> | metric | SMALL initial | SMALL rebalance | LARGE initial | LARGE rebalance | |---|---|---|---|---| | stateful active tasks per member (max-min) | 0.747 | 0.738 | 1.000 | 1.022 → 1.037 | | stateless active tasks per member (max-min) | 0.887 | 0.850 | 1.767 | 1.644 → 1.652 | | active tasks per member (max-min) | 0.870 | 0.855 → 0.856 | 1.000 | 1.185 → 1.200 | | stateful active tasks above even share | 0.000 | 0.000 | 0.000 | 0.000 | | stateless active tasks above even share | 0.097 | 0.060 → 0.055 | 0.333 | 0.385 → 0.378 | | active tasks above even share | 0.000 | 0.000 | 0.000 | 0.000 | | total tasks per member (max-min) | 1.080 → 1.130 | 1.419 → 1.453 | 1.233 → 1.733 | 1.467 → 1.585 | | process load (max-min) | 0.651 → 0.715 | 1.042 → 1.081 | 0.774 → 1.282 | 0.745 → 1.048 | | standby load per process (max-min) | 0.472 → 0.533 | 0.824 → 0.881 | 0.677 → 1.168 | 0.652 → 0.981 | | subtopology skew above unavoidable (max-min) | 0.233 | 0.455 → 0.464 | 1.000 | 1.496 | | subtopologies skewed above unavoidable | 0.237 | 0.575 → 0.601 | 2.867 | 4.237 → 4.311 | | distinct tag values [region] (achieved / achievable) | 0.627 → 0.789 | 0.620 → 0.771 | 0.555 → 0.783 | 0.520 → 0.758 | | distinct tag values [zone] (achieved / achievable) | 0.579 → 0.709 | 0.562 → 0.697 | 0.578 → 0.790 | 0.549 → 0.718 | | distinct tag values [rack] (achieved / achievable) | 0.559 → 0.665 | 0.590 → 0.717 | n/a | n/a | | ideal distribution through [region] | 0.546 → 0.737 | 0.520 → 0.698 | 0.515 → 0.743 | 0.465 → 0.706 | | ideal distribution through [zone] | 0.311 → 0.480 | 0.257 → 0.428 | 0.329 → 0.619 | 0.294 → 0.554 | | ideal distribution through [rack] | 0.126 → 0.266 | 0.134 → 0.295 | n/a | n/a | | all replicas on one tag value [region] | 0.301 → 0.174 | 0.294 → 0.177 | 0.410 → 0.193 | 0.431 → 0.210 | | all replicas on one tag value [zone] | 0.348 → 0.229 | 0.348 → 0.221 | 0.396 → 0.192 | 0.414 → 0.253 | | all replicas on one tag value [rack] | 0.339 → 0.227 | 0.317 → 0.200 | n/a | n/a | | stateful active tasks moved | n/a | 0.470 → 0.466 | n/a | 4.548 | | standby tasks moved | n/a | 0.418 → 0.599 | n/a | 9.607 → 20.133 | | replicas placed without state | n/a | 1.419 → 1.604 | n/a | 17.889 → 28.422 | | stateful active tasks kept on process (1.0 = all) | n/a | 0.953 → 0.954 | n/a | 0.991 → 0.992 | | stateful active tasks kept on member (1.0 = all) | n/a | 0.968 | n/a | 0.992 | | stateless active tasks kept on member (1.0 = all) | n/a | 0.912 | n/a | 0.977 → 0.978 | | standby tasks kept on process (1.0 = all) | n/a | 0.921 → 0.889 | n/a | 0.961 → 0.942 | | standby tasks kept on member (1.0 = all) | n/a | 0.912 → 0.873 | n/a | 0.957 → 0.931 | | rounds to converge | 1.020 → 1.030 | 1.024 → 1.021 | 1.200 → 1.517 | 1.496 → 1.632 | | not converged | 0.000 | 0.000 | 0.000 → 0.033 | 0.000 → 0.015 | | process load (max-min) (tag-blind) | 0.649 | 1.049 → 1.078 | 0.774 | 0.745 → 0.987 | | standby load per process (max-min) (tag-blind) | 0.466 | 0.827 → 0.863 | 0.677 | 0.652 → 0.886 | | distinct tag values [region] (achieved / achievable) (tag-blind) | 0.627 | 0.620 → 0.720 | 0.555 | 0.520 → 0.745 | | distinct tag values [zone] (achieved / achievable) (tag-blind) | 0.579 | 0.562 → 0.651 | 0.578 | 0.549 → 0.704 | | distinct tag values [rack] (achieved / achievable) (tag-blind) | 0.559 | 0.590 → 0.689 | n/a | n/a | | all replicas on one tag value [region] (tag-blind) | 0.301 | 0.294 → 0.222 | 0.410 | 0.431 → 0.222 | | all replicas on one tag value [zone] (tag-blind) | 0.348 | 0.348 → 0.266 | 0.396 | 0.414 → 0.267 | | all replicas on one tag value [rack] (tag-blind) | 0.339 | 0.317 → 0.223 | n/a | n/a | | standby tasks moved (tag-blind) | n/a | 0.421 → 0.460 | n/a | 9.607 → 13.659 | </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
