Jacek Lewandowski created SPARK-27220: -----------------------------------------
Summary: Remove Yarn specific leftover from CoarseGrainedSchedulerBackend Key: SPARK-27220 URL: https://issues.apache.org/jira/browse/SPARK-27220 Project: Spark Issue Type: Task Components: Spark Core, YARN Affects Versions: 2.4.0, 2.3.3, 2.2.3, 2.1.3, 2.0.2 Reporter: Jacek Lewandowski {{CoarseGrainedSchedulerBackend}} has the following field: {code:scala} // The num of current max ExecutorId used to re-register appMaster @volatile protected var currentExecutorIdCounter = 0 {code} which is then updated: {code:scala} case RegisterExecutor(executorId, executorRef, hostname, cores, logUrls) => ... // This must be synchronized because variables mutated // in this block are read when requesting executors CoarseGrainedSchedulerBackend.this.synchronized { executorDataMap.put(executorId, data) if (currentExecutorIdCounter < executorId.toInt) { currentExecutorIdCounter = executorId.toInt } ... {code} However it is never really used in {{CoarseGrainedSchedulerBackend}}. Its only usage is in Yarn-specific code. It should be moved to Yarn then because {{executorId}} is a {{String}} and there are really no guarantees that it is always an integer. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org