phillipleblanc commented on code in PR #1893:
URL:
https://github.com/apache/datafusion-ballista/pull/1893#discussion_r3611672002
##########
ballista/scheduler/src/config.rs:
##########
@@ -34,6 +34,16 @@ use datafusion_proto::physical_plan::PhysicalExtensionCodec;
use std::fmt::Display;
use std::sync::Arc;
+/// Callback invoked when new work becomes available for executors.
+///
+/// This is called after:
+/// - A job is submitted and tasks are ready to be scheduled
+/// - Tasks complete and new stages become runnable
+///
+/// This allows external systems to notify executors to poll immediately
+/// rather than waiting for their next poll interval.
+pub type OnWorkAvailableFn = Arc<dyn Fn(&str) + Send + Sync>;
Review Comment:
Replaced the `&str` with a `WorkAvailableReason` enum (`JobSubmitted {
job_id }` / `NewStagesRunnable { job_id }`), so consumers get structured data
instead of parsing strings.
It also needs to stay `Arc` because `SchedulerConfig` derives `Clone` and
`Box<dyn Fn>` isn't cloneable. We could drop the Clone requirement if we want
to keep it as Box.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
