[
https://issues.apache.org/jira/browse/SPARK-11155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14962151#comment-14962151
]
Imran Rashid commented on SPARK-11155:
--------------------------------------
[~kayousterhout] I'm referring to the http endpoints that expose the data in
the UI as json http://spark.apache.org/docs/latest/monitoring.html#rest-api.
Eg., if you run any job in a spark-shell, you can go to:
http://localhost:4040/api/v1/applications/Spark%20shell/stages
The json is produced from
[o.a.s.status.api.v1.StageData|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/api.scala#L110],
which doesn't have the right fields.
> Stage summary json should include stage duration
> -------------------------------------------------
>
> Key: SPARK-11155
> URL: https://issues.apache.org/jira/browse/SPARK-11155
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Reporter: Imran Rashid
> Priority: Minor
> Labels: Starter
>
> The json endpoint for stages doesn't include information on the stage
> duration that is present in the UI. This looks like a simple oversight, they
> should be included. eg., the metrics should be included at
> {{api/v1/applications/<appId>/stages}}. The missing metrics are
> {{submissionTime}} and {{completionTime}} (and whatever other metrics come
> out of the discussion on SPARK-10930)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
