Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19942#discussion_r156664176
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -564,6 +564,14 @@ class SparkConf(loadDefaults: Boolean) extends
Cloneable with Logging with Seria
val encryptionEnabled = get(NETWORK_ENCRYPTION_ENABLED) ||
get(SASL_ENCRYPTION_ENABLED)
require(!encryptionEnabled || get(NETWORK_AUTH_ENABLED),
s"${NETWORK_AUTH_ENABLED.key} must be enabled when enabling
encryption.")
+
+ val executorTimeoutThreshold =
getTimeAsSeconds("spark.network.timeout", "120s")
+ val executorHeartbeatInterval =
getTimeAsSeconds("spark.executor.heartbeatInterval", "10s")
+ // If spark.executor.heartbeatInterval bigger than
spark.network.timeout,
+ // it will almost always cause ExecutorLostFailure. See SPARK-22754.
+ require(executorTimeoutThreshold > executorHeartbeatInterval, s"The
value of " +
--- End diff --
No need for 's', and you're missing an open quote in the next line. You
could print the value of these params inline too.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
