LuciferYang commented on a change in pull request #30547:
URL: https://github.com/apache/spark/pull/30547#discussion_r533093278
##########
File path: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala
##########
@@ -80,7 +80,7 @@ private[spark] class HeartbeatReceiver(sc: SparkContext,
clock: Clock)
// executor ID -> timestamp of when the last heartbeat from this executor
was received
private val executorLastSeen = new HashMap[String, Long]
- private val executorTimeoutMs =
sc.conf.get(config.STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT)
+ private val executorTimeoutMs =
Utils.blockManagerHeartbeatTimeoutAsMs(sc.conf)
Review comment:
definition `STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT` from
```
private[spark] val STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT =
ConfigBuilder("spark.storage.blockManagerHeartbeatTimeoutMs")
.version("0.7.0")
.withAlternative("spark.storage.blockManagerSlaveTimeoutMs")
.timeConf(TimeUnit.MILLISECONDS)
.createWithDefaultString(Network.NETWORK_TIMEOUT.defaultValueString)
```
to
```
private[spark] val STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT =
ConfigBuilder("spark.storage.blockManagerHeartbeatTimeoutMs")
.version("0.7.0")
.withAlternative("spark.storage.blockManagerSlaveTimeoutMs")
.fallbackConf(Network.NETWORK_TIMEOUT)
```
?
If so, the TimeUnit of `STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT` will change
from `TimeUnit.MILLISECONDS` to `TimeUnit.SECONDS`, and the default value will
change from `120000` to `120`,
This does not match the "Ms" unit described in
`spark.storage.blockManagerHeartbeatTimeoutMs`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
