[
https://issues.apache.org/jira/browse/SPARK-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-6702:
---------------------------------
Description:
Since the Streaming Tab was added to the Spark UI in Spark 1.0, its has proven
to be a defacto way to debug Spark Streaming applications at the time of
development. However, there is much more we can do to help developers debug
their streaming applications. Specifically we would like to show the details of
individual batches and link them to the jobs that ran for that batch (similar
to job details showing stages in the job). Furthermore we would also like show
timelines to see trends over time for processing times data rates etc.
This is the design doc that shows the proposed new Streaming tab in the Spark
UI.
https://docs.google.com/document/d/1-ZjvQ_2thWEQkTxRMHrVdnEI57XTi3wZEBUoqrrDg5c/edit?usp=sharing
was:Since the Streaming Tab was added to the Spark UI in Spark 1.0, its has
proven to be a defacto way to debug Spark Streaming applications at the time of
development. However, there is much more we can do to help developers debug
their streaming applications. Specifically we would like to show the details of
individual batches and link them to the jobs that ran for that batch (similar
to job details showing stages in the job). Furthermore we would also like show
timelines to see trends over time for processing times data rates etc.
> Update the Streaming Tab in Spark UI to show more batch information
> -------------------------------------------------------------------
>
> Key: SPARK-6702
> URL: https://issues.apache.org/jira/browse/SPARK-6702
> Project: Spark
> Issue Type: Improvement
> Components: Streaming, Web UI
> Affects Versions: 1.3.1
> Reporter: Tathagata Das
> Assignee: Tathagata Das
>
> Since the Streaming Tab was added to the Spark UI in Spark 1.0, its has
> proven to be a defacto way to debug Spark Streaming applications at the time
> of development. However, there is much more we can do to help developers
> debug their streaming applications. Specifically we would like to show the
> details of individual batches and link them to the jobs that ran for that
> batch (similar to job details showing stages in the job). Furthermore we
> would also like show timelines to see trends over time for processing times
> data rates etc.
> This is the design doc that shows the proposed new Streaming tab in the Spark
> UI.
> https://docs.google.com/document/d/1-ZjvQ_2thWEQkTxRMHrVdnEI57XTi3wZEBUoqrrDg5c/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
