squito commented on a change in pull request #23616: [SPARK-26688][YARN]
Provide configuration of initially blacklisted YARN nodes
URL: https://github.com/apache/spark/pull/23616#discussion_r255695395
##########
File path:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTrackerSuite.scala
##########
@@ -137,4 +144,41 @@ class YarnAllocatorBlacklistTrackerSuite extends
SparkFunSuite with Matchers
verify(amClientMock).updateBlacklist(Arrays.asList("host4"),
Collections.emptyList())
assert(yarnBlacklistTracker.isAllNodeBlacklisted === true)
}
+
+ test("test with initial blacklisting") {
+ sparkConf.set(YARN_EXECUTOR_INITIAL_BLACKLISTED_NODES, Seq("initial1",
"initial2"))
+ val yarnBlacklistTracker = createYarnAllocatorBlacklistTracker(sparkConf)
+ yarnBlacklistTracker.setSchedulerBlacklistedNodes(Set())
Review comment:
Just thinking aloud -- could this just be added to one of the other tests,
eg. "combining scheduler and allocation blacklist"? After the first check, the
rest of this test doesn't seem distinct from the others, I'm not sure its worth
having a separate test for it (harder to maintain redundant tests down the road
if behavior changes, etc.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
