attilapiros edited a comment on pull request #31513: URL: https://github.com/apache/spark/pull/31513#issuecomment-785738931
Rebased on top of the master and new test is added. There is new commit e1434cd which changes the flag `hasPendingPods` to `numOutstandingPods` which used in asserts in all the existing tests, all tests passed in that commit: ``` $ git checkout e1434cd ... $ NOLINT_ON_COMPILE=1 ./build/sbt -Pyarn -Phive -Pkubernetes -Pkubernetes-integration-test ... sbt:spark-kubernetes> testOnly *.ExecutorPodsAllocatorSuite [warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings. [warn] multiple main classes detected: run 'show discoveredMainClasses' to see the list [info] compiling 3 Scala sources to /Users/attilazsoltpiros/git/attilapiros/spark/resource-managers/kubernetes/core/target/scala-2.12/classes ... [info] compiling 3 Scala sources and 1 Java source to /Users/attilazsoltpiros/git/attilapiros/spark/core/target/scala-2.12/test-classes ... [info] compiling 2 Scala sources to /Users/attilazsoltpiros/git/attilapiros/spark/resource-managers/kubernetes/core/target/scala-2.12/test-classes ... [info] ExecutorPodsAllocatorSuite: [info] - Initially request executors in batches. Do not request another batch if the first has not finished. (65 milliseconds) [info] - Request executors in batches. Allow another batch to be requested if all pending executors start running. (40 milliseconds) [info] - When a current batch reaches error states immediately, re-request them on the next batch. (16 milliseconds) [info] - When an executor is requested but the API does not report it in a reasonable time, retry requesting that executor. (8 milliseconds) [info] - SPARK-28487: scale up and down on target executor count changes (34 milliseconds) [info] - SPARK-34334: correctly identify timed out pending pod requests as excess (9 milliseconds) [info] - SPARK-33099: Respect executor idle timeout configuration (11 milliseconds) [info] - SPARK-33288: multiple resource profiles (21 milliseconds) [info] - SPARK-33262: pod allocator does not stall with pending pods (8 milliseconds) [info] ScalaTest [info] Run completed in 2 seconds, 529 milliseconds. [info] Total number of tests run: 9 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 9, failed 0, canceled 0, ignored 0, pending 0 [info] All tests passed. ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
