1fanwang opened a new pull request, #2393: URL: https://github.com/apache/datafusion-ballista/pull/2393
Fixes https://github.com/apache/datafusion-ballista/issues/2356 An executor can hang after receiving SIGTERM or Ctrl-C while a task is active. The task eventually finishes, but `TasksDrainedFuture` returned `Pending` without registering a waker, so shutdown never checked the task map again. The drain future now registers an `AtomicWaker` before checking active tasks. Cancellation leaves the task registered until `execute_query_stage` finishes its cleanup. Removing the last task then wakes the drain future and lets shutdown continue. ### Testing The regression drives the real task registration and cancellation path. On `7e939fe26c8cfacc41ed16a656704d69c94ea380`, with only the regression applied: ```text $ cargo test -p ballista-executor executor::test::test_task_cancellation --lib -- --nocapture thread 'executor::test::test_task_cancellation' panicked: assertion failed: tokio::time::timeout(Duration::from_secs(1), tasks_drained).await.is_ok() test executor::test::test_task_cancellation ... FAILED test result: FAILED. 0 passed; 1 failed; 52 filtered out ``` On this branch: ```text $ cargo test -p ballista-executor executor::test::test_task_cancellation --lib -- --nocapture test executor::test::test_task_cancellation ... ok test result: ok. 1 passed; 0 failed; 52 filtered out $ cargo test -p ballista-executor --lib test result: ok. 53 passed; 0 failed; 0 ignored; 0 measured ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
