[
https://issues.apache.org/jira/browse/SPARK-33557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-33557:
---------------------------------
Fix Version/s: 3.0.2
> spark.storage.blockManagerSlaveTimeoutMs default value does not follow
> spark.network.timeout value when the latter was changed
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-33557
> URL: https://issues.apache.org/jira/browse/SPARK-33557
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.0.0, 3.0.1
> Reporter: Ohad
> Assignee: Yang Jie
> Priority: Minor
> Fix For: 3.0.2, 3.1.0
>
>
> According to the documentation "spark.network.timeout" is the default timeout
> for "spark.storage.blockManagerSlaveTimeoutMs" which implies that when the
> user sets "spark.network.timeout" the effective value of
> "spark.storage.blockManagerSlaveTimeoutMs" should also be changed if it was
> not specifically changed.
> However this is not the case since the default value of
> "spark.storage.blockManagerSlaveTimeoutMs" is always the default value of
> "spark.network.timeout" (120s)
>
> "spark.storage.blockManagerSlaveTimeoutMs" is defined in the package object
> of "org.apache.spark.internal.config" as follows:
> {code:java}
> private[spark] val STORAGE_BLOCKMANAGER_SLAVE_TIMEOUT =
> ConfigBuilder("spark.storage.blockManagerSlaveTimeoutMs")
> .version("0.7.0")
> .timeConf(TimeUnit.MILLISECONDS)
> .createWithDefaultString(Network.NETWORK_TIMEOUT.defaultValueString)
> {code}
> So it seems like the its default value is indeed "fixed" to
> "spark.network.timeout" default value.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
