tgravescs commented on code in PR #39127:
URL: https://github.com/apache/spark/pull/39127#discussion_r1109108543
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala:
##########
@@ -563,6 +565,7 @@ private[yarn] class YarnAllocator(
}
}
+ allocatorNodeHealthTracker.setSchedulerExcludedNodes(excludeNodes)
Review Comment:
this feels like a weird way to do this based on the
YarnAllocatorNodeHealthTracker already reads the excludeNode and has it as a
private variable. This api was meant for excluding nodes that had dynamic
failures... ie the scheduler excluded them. These are not excluded by the
scheduler but by user manually.
what if we just have it call refreshExcludedNodes when the
YarnAllocatorNodeHealthTracker is created? Unless you are trying to
dynamically change this config which it wasn't really meant for.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
