[
https://issues.apache.org/jira/browse/SPARK-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or resolved SPARK-13269.
-------------------------------
Resolution: Fixed
Assignee: Wenchen Fan
Fix Version/s: 2.0.0
> Expose more executor stats in stable status API
> -----------------------------------------------
>
> Key: SPARK-13269
> URL: https://issues.apache.org/jira/browse/SPARK-13269
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Andrew Or
> Assignee: Wenchen Fan
> Fix For: 2.0.0
>
>
> Currently the stable status API is quite limited; it exposes only a small
> subset of the things exposed by JobProgressListener. It is useful for very
> high level querying but falls short when the developer wants to build an
> application on top of Spark with more integration.
> In this issue I propose that we expose at least two things:
> - Which executors are running tasks, and
> - Which executors cached how much in memory and on disk
> The goal is not to expose exactly these two things, but to expose something
> that would allow the developer to learn about them. These concepts are very
> much fundamental in Spark's design so there's almost no chance that they will
> go away in the future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
