[
https://issues.apache.org/jira/browse/SPARK-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13959494#comment-13959494
]
Patrick Wendell commented on SPARK-1411:
----------------------------------------
Ah, got it SPARK-1337
> When using spark.ui.retainedStages=n only the first n stages are kept, not
> the most recent.
> -------------------------------------------------------------------------------------------
>
> Key: SPARK-1411
> URL: https://issues.apache.org/jira/browse/SPARK-1411
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 0.9.0
> Environment: Ubuntu 12.04 LTS Precise
> 4 Nodes,
> 16 cores each,
> 196GB RAM each.
> Reporter: Davis Shepherd
> Attachments: Screen Shot 2014-04-03 at 5.35.00 PM.png
>
>
> For any long running job with many stages, the web ui only shows the first n
> stages of the job (where spark.ui.retainedStages=n). The most recent stages
> are immediately dropped and are only visible for a brief time. This renders
> the UI pretty useless after a pretty short amount of time for a long running
> non-streaming job. I am unsure as to whether similar results appear for
> streaming jobs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
