andygrove opened a new issue, #2596: URL: https://github.com/apache/datafusion-ballista/issues/2596
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** [`QueryStageScheduler::on_receive` (query_stage_scheduler.rs#L169-L660)](https://github.com/apache/datafusion-ballista/blob/ea1126c0649b2bc08e1e233b1972b64318821c16/ballista/scheduler/src/scheduler_server/query_stage_scheduler.rs#L169-L660) is the scheduler's central event loop and a single 415-line function (cyclomatic complexity 58, cognitive 104), changed in 22 commits over the last 12 months. - A ~100-line `#[cfg(feature = "rest-api")]` event-log pre-pass matches on the same event variants as the main dispatch, and repeats the terminal-event pattern (`event_log_graph`, `append_final`, `finish_job`) three times. - The main match inlines each arm: the JobQueued arm spawns the submit task with subscriber failure notification, the TaskUpdating arm does the vcore refund, and the ExecutorLost arm (about L540-L650) embeds a nested spawned grace-period task that re-reads job status, reaching nesting depth 5. Changes here are risky and hard to review because unrelated events share one function. **Describe the solution you'd like** - Keep `on_receive` as a thin dispatcher plus the timing wrapper, and move each non-trivial arm into a named method (`on_job_queued`, `on_job_submitted`, `on_job_finished`, `on_job_running_failed`, `on_task_updating`, `on_executor_lost`, ...). - Move the rest-api pre-pass into `record_event_log(&self, &event)` with a `finish_event_log(job_id, ...)` helper for the three terminal events. - Extract the no-executor grace-period task into `schedule_no_executor_check(...)`. **Describe alternatives you've considered** Leave it as is. **Additional context** Line references are pinned to commit ea1126c0649b. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
