[
https://issues.apache.org/jira/browse/SPARK-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364254#comment-14364254
]
Saisai Shao commented on SPARK-6304:
------------------------------------
Hi [~msoutier], the reason to remove these two configurations, especially
"spark.driver.port" is that: SparkContext itself will randomly choose a port
and set it to configuration even user didn't set it, next time after
application is recovered, previous configuration "spark.driver.port" need to
remove and let SparkContext itself to randomly choose again and set into the
SparkConf. So that's why checkpoint need to remove these two configurations.
> Checkpointing doesn't retain driver port
> ----------------------------------------
>
> Key: SPARK-6304
> URL: https://issues.apache.org/jira/browse/SPARK-6304
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.2.1
> Reporter: Marius Soutier
>
> In a check-pointed Streaming application running on a fixed driver port, the
> setting "spark.driver.port" is not loaded when recovering from a checkpoint.
> (The driver is then started on a random port.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
