[
https://issues.apache.org/jira/browse/SPARK-11457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-11457.
------------------------------------
Resolution: Fixed
Assignee: Saisai Shao
Fix Version/s: 1.6.0
> Yarn AM proxy filter configuration should be reloaded when recovered from
> checkpoint
> ------------------------------------------------------------------------------------
>
> Key: SPARK-11457
> URL: https://issues.apache.org/jira/browse/SPARK-11457
> Project: Spark
> Issue Type: Bug
> Components: Streaming, YARN
> Affects Versions: 1.5.1
> Reporter: Saisai Shao
> Assignee: Saisai Shao
> Fix For: 1.6.0
>
>
> Currently Yarn AM proxy filter configuration is recovered from checkpoint
> file when Spark Streaming application is restarted, which will lead to some
> unwanted behaviors:
> 1. Wrong RM address if RM is redeployed from failure.
> 2. Wrong proxyBase, since app id is updated, old app id for proxyBase is
> wrong.
> So instead of recovering from checkpoint file, these configurations should be
> reloaded each time when app started.
> This problem only exists in Yarn cluster mode, for Yarn client mode, these
> configurations will be updated with RPC message {{AddWebUIFilter}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
