andygrove commented on PR #1685: URL: https://github.com/apache/datafusion-ballista/pull/1685#issuecomment-6723302305
Hey @abhinavgautam01, sorry for the very long wait on this one. That's on me. In the meantime #2669 landed and covers the standalone side of this: the scheduler advertises the port it actually bound, and `ballista.standalone.task_scheduling_policy` = `push-staged` runs the executor through `executor_server::startup_with_incoming` with its gRPC listener already bound. That overlaps a lot with the executor and scheduler changes here, and the branch now conflicts with `main` in most files. The part that's still really valuable is running the existing client suites under both policies, which is what #1463 asks for. `main` only has `standalone_push_staged.rs`, which runs three queries. Would you be up for rebasing onto `main` and trimming this to just the test changes? The `#[values(...)]` setup for pull and push from killzoner's suggestion could build its contexts with the new setting through `ballista_state(...)` in `tests/common/mod.rs`. Then there should be no need to touch `standalone.rs` or `executor_server.rs` at all. Happy to review quickly once it's updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
