attilapiros opened a new pull request #31790: URL: https://github.com/apache/spark/pull/31790
This is a work in progress PR as I first would like test this solution extensively on a real k8s cluster not only with unit tests. ### What changes were proposed in this pull request? Making upscaling more progressive to eagerly go up to the configured allocation batch size. ### Why are the changes needed? The current solution will stops requesting PODS when even one POD request is active form the current batch. After this PR allocation goes up to batch size as early as possible. My reasoning is the following: when POD allocation takes a long time then let's request as soon as possible, excess requests will be deleted anyway by downscaling. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing unit test. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
