Github user ssaavedra commented on a diff in the pull request:
https://github.com/apache/spark/pull/19427#discussion_r148945009
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala ---
@@ -62,6 +63,7 @@ class Checkpoint(ssc: StreamingContext, val
checkpointTime: Time)
val newSparkConf = new SparkConf(loadDefaults =
false).setAll(sparkConfPairs)
.remove("spark.driver.host")
+ .remove("spark.driver.bindAddress")
--- End diff --
Yes. If it is not set in the new run, it should still be meaningless
anyway. It makes sense to know this property on the subsequent calls to
spark-submit. If we are resuming a checkpoint it means we are re-submitting
work, but it may be run in a different cluster configuration, and thus we may
want to change the bindAddress or this different configuration may even wish to
rely on falling back to the `spark.driver.host` configuration. In any case, it
should make no sense to keep the old setting, unless we are running on a static
configuration, in which case it is not a caveat to remove this, as the
command-line to re-launch the job can still re-populate the property if it
needs to keep being the same.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
