Ohad created SPARK-33557: ---------------------------- Summary: spark.storage.blockManagerSlaveTimeoutMs default value does not follow spark.network.timeout value when the latter was changed Key: SPARK-33557 URL: https://issues.apache.org/jira/browse/SPARK-33557 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 3.0.1, 3.0.0 Reporter: Ohad
According to the documentation "spark.network.timeout" is the default timeout for "spark.storage.blockManagerSlaveTimeoutMs" which implies that when the user sets "spark.network.timeout" the effective value of "spark.storage.blockManagerSlaveTimeoutMs" should also be changed if it was not specifically changed. However this is not the case since the default value of "spark.storage.blockManagerSlaveTimeoutMs" is always the default value of "spark.network.timeout" (120s) "spark.storage.blockManagerSlaveTimeoutMs" is defined in the package object of "org.apache.spark.internal.config" as follows: {code:java} private[spark] val STORAGE_BLOCKMANAGER_SLAVE_TIMEOUT = ConfigBuilder("spark.storage.blockManagerSlaveTimeoutMs") .version("0.7.0") .timeConf(TimeUnit.MILLISECONDS) .createWithDefaultString(Network.NETWORK_TIMEOUT.defaultValueString) {code} So it seems like the its default value is indeed "fixed" to "spark.network.timeout" default value. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org