[
https://issues.apache.org/jira/browse/SPARK-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032419#comment-15032419
]
Steve Loughran commented on SPARK-5106:
---------------------------------------
The discussion on the PR has essentially concluded that this idea "has issues",
especially in some deployments.
However, it does relate to SPARK-7889 and the history server refresh problem:
it doesn't pick up changed histories.
What if the way the UI updated itself was pluggable? Then the single-instance
history view could be auto-updating, but the history server would have
different policies
# don't update a completed app, ever
# update a running application, but only at a slower rate
This would avoid the problem which my current PR for SPARK-7889 has: even
though the history server can see if something has changed, the self-contained
UI can't pick up those changes
> Make Web UI automatically refresh/update displayed data
> -------------------------------------------------------
>
> Key: SPARK-5106
> URL: https://issues.apache.org/jira/browse/SPARK-5106
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 1.2.0
> Reporter: Ryan Williams
>
> My (and presumably others') experience monitoring Spark jobs currently
> consists of repeatedly ⌘R'ing various pages of the web UI to view
> ever-fresher data about how many tasks have succeeded / failed, how much
> spillage is happening, etc., which is tedious.
> Particularly unfortunate is the "one refresh over the line" problem where,
> just as things are getting interesting, the job itself fails or finishes, and
> after refreshing the page all data disappears.
> It would be good if the web UI updated the data it was displaying
> automatically.
> One hacky way to achieve this would be to have it automatically refresh the
> page, though this still risks losing everything when the job finishes.
> A better long-term solution would be to have the UI poll for (or have pushed
> to it) updates to the data it is displaying.
> Either way, some way to toggle this functionality on or off is probably
> warranted as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
