FridaPolished opened a new pull request #32304:
URL: https://github.com/apache/spark/pull/32304
**What changes were proposed in this pull request?**
The original Spark UI on SHS does not have a specific way to explore
accumulators information, this data is scattered across stages which makes it
difficult to access it.
The accumulators information is located on the Stage Page, but this column
is hard to read since it groups all the accumulators names and values for a
specific task. This is one of the reasons why understanding the changes of
counters from a job can become a difficult task.
The proposed change creates an explore page for Accumulators on the SparkUI.
The information will be located under a new tab that has an overview page with
links to check for more details about the accumulators information by a
particular name or stage.
[Community discussion on accumulators
page](http://apache-spark-developers-list.1001551.n3.nabble.com/accumulators-td8814.html)
**Why are the changes needed?**
A page that allows users to explore accumulators information is a helpful
feature for users because currently the information is scattered across stages
and the display on the Stages Page is difficult to read. This page offers easy
access to accumulators information in detail which translates into a better
user experience and an increased velocity to solve problems and track
accumulator changes.
**Does this PR introduce user-facing change?**
Yes. A new tab will be added to the Spark UI and three new pages will
dependent on it.
<img width="889" alt="Screen Shot 2021-04-09 at 3 05 39 PM"
src="https://user-images.githubusercontent.com/45573418/115795081-15b31300-a384-11eb-88c2-56d7454faee5.png";>
**How was this patch tested?**
This patch was tested manually by using real spark event logs from a Spark
History Server on a production cluster. I also used unit testing to check for
contents inside the new data structure used for the Accumulators Overview page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/45573418/115795081-15b31300-a384-11eb-88c2-56d7454faee5.png"](https://user-images.githubusercontent.com/45573418/115795081-15b31300-a384-11eb-88c2-56d7454faee5.png"){kind=link}