[
https://issues.apache.org/jira/browse/SPARK-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021008#comment-14021008
]
Mridul Muralidharan commented on SPARK-2064:
--------------------------------------------
Unfortunately OOM is a very big issue for us since application master is single
point of failure when running in yarn.
Particularly when memory is constrained and vigorously enforced by the yarn
containers (requiring higher overheads to be specified reducing usable memory
even further.
Given this, and given the fair churn already for executor containers, I am
hesitant about features which add to the memory footprint for UI even further.
The cumulative impact of ui is nontrivial as I mentioned before. This, for
example, would require 1-8% of master memory when there is reasonable churn for
long running jobs (30 hours) on reasonable number of executors (200-300).
> web ui should not remove executors if they are dead
> ---------------------------------------------------
>
> Key: SPARK-2064
> URL: https://issues.apache.org/jira/browse/SPARK-2064
> Project: Spark
> Issue Type: Sub-task
> Reporter: Reynold Xin
>
> We should always show the list of executors that have ever been connected,
> and add a status column to mark them as dead if they have been disconnected.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
