andygrove opened a new issue, #2600: URL: https://github.com/apache/datafusion-ballista/issues/2600
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** In [`ballista/scheduler/src/cluster/mod.rs`](https://github.com/apache/datafusion-ballista/blob/ea1126c0649b2bc08e1e233b1972b64318821c16/ballista/scheduler/src/cluster/mod.rs), [`bind_task_bias` (L522-L573)](https://github.com/apache/datafusion-ballista/blob/ea1126c0649b2bc08e1e233b1972b64318821c16/ballista/scheduler/src/cluster/mod.rs#L522-L573) and [`bind_task_round_robin` (L575-L631)](https://github.com/apache/datafusion-ballista/blob/ea1126c0649b2bc08e1e233b1972b64318821c16/ballista/scheduler/src/cluster/mod.rs#L575-L631) share about 30 lines of identical scaffolding: the all-budgets-zero early return, the descending sort by vcores, iterating running jobs and skipping non-Running ones, taking the graph write lock, the `fetch_running_stage` loop with `if_skip` / blacklist handling, and the call to `bind_one`. Only the inner executor-selection loop differs, and the two log messages already differ by hand. `bind_task_round_robin` also has a hand-rolled circular scan (about L606-L613) with a `scanned` counter and modular index arithmetic at nesting depth 5. It only avoids indexing an empty `budgets` because of the earlier all-zero check. **Describe the solution you'd like** - Introduce a shared driver, e.g. `bind_tasks_with(budgets, running_jobs, if_skip, strategy)`, that owns the preamble, job iteration, graph locking and skip/blacklist logic, parameterised by a small `BindStrategy` trait or closure for executor selection. - Extract `next_with_capacity(budgets, start) -> Option<usize>` for the circular scan and unit-test it directly (all zero, wraparound, single executor, empty). - `bind_task_bias` and `bind_task_round_robin` become thin wrappers. **Describe alternatives you've considered** None. **Additional context** Line references are pinned to commit ea1126c0649b. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
