villebro commented on PR #2631: URL: https://github.com/apache/datafusion-ballista/pull/2631#issuecomment-6303933707
@milenkovicm with shared state that should be fairly easy to achieve. If we do go down that route, the scheduler would hand its jobs over to another scheduler instead of waiting for them to finish, and the timeout would only limit how long it waits for another scheduler to become available. With no other scheduler available it would fall back to what this PR does, so the wait for active jobs is the only part that changes. The rest of the drain (refusing new jobs, failing /readyz, closing connections) stays. A job could then outlive any one scheduler, so we'd also want a query timeout, with a default and a max on the scheduler and a per-query override within those bounds. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
