gaborgsomogyi commented on a change in pull request #24935: [SPARK-28005][YARN]
Remove unnecessary log from SparkRackResolver
URL: https://github.com/apache/spark/pull/24935#discussion_r297128415
##########
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/SparkRackResolver.scala
##########
@@ -70,18 +70,20 @@ private[spark] class SparkRackResolver(conf:
Configuration) extends Logging {
val nodes = new ArrayBuffer[Node]
// dnsToSwitchMapping is thread-safe
val rNameList = dnsToSwitchMapping.resolve(hostNames.toList.asJava).asScala
- if (rNameList == null || rNameList.isEmpty) {
+ if (rNameList == null) {
hostNames.foreach(nodes += new NodeBase(_, NetworkTopology.DEFAULT_RACK))
logInfo(s"Got an error when resolving hostNames. " +
s"Falling back to ${NetworkTopology.DEFAULT_RACK} for all")
} else {
- for ((hostName, rName) <- hostNames.zip(rNameList)) {
- if (Strings.isNullOrEmpty(rName)) {
- nodes += new NodeBase(hostName, NetworkTopology.DEFAULT_RACK)
- logDebug(s"Could not resolve $hostName. " +
- s"Falling back to ${NetworkTopology.DEFAULT_RACK}")
- } else {
- nodes += new NodeBase(hostName, rName)
+ if (rNameList.nonEmpty) {
+ for ((hostName, rName) <- hostNames.zip(rNameList)) {
Review comment:
Ooooh gosh! This PR is not something which I'm telling tales to my children
:)
I was sooo focused on the if condition which wrapped the log that I've mixed
up `rNameList` and `hostNames`. This case I agree on having an early return.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
