[
https://issues.apache.org/jira/browse/SPARK-27394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-27394.
------------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
> The staleness of UI may last minutes or hours when no tasks start or finish
> ---------------------------------------------------------------------------
>
> Key: SPARK-27394
> URL: https://issues.apache.org/jira/browse/SPARK-27394
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 2.4.0, 2.4.1
> Reporter: Shixiong Zhu
> Assignee: Shixiong Zhu
> Priority: Major
> Fix For: 3.0.0
>
>
> Run the following codes on a cluster that has at least 2 cores.
> {code}
> sc.makeRDD(1 to 1000, 1000).foreach { i =>
> Thread.sleep(300000)
> }
> {code}
> The jobs page will just show one running task.
> This is because when the second task event calls
> "AppStatusListener.maybeUpdate" for a job, it will just ignore since the gap
> between two events is smaller than `spark.ui.liveUpdate.period`.
> After the second task event, in the above case, because there won't be any
> other task events, the Spark UI will be always stale until the next task
> event gets fired (after 300 seconds).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
