I feel this pain frequently Something more interactive would be great On Wed, 6 Sep 2023 at 4:34 PM, Santosh Pingale <santosh.ping...@adyen.com.invalid> wrote:
> Hey community > > Spark UI with the plan visualisation is an excellent resource for finding > out crucial information about how your application is doing and what parts > of the execution can still be optimized to fulfill time/resource > constraints. > > The graph in its current form is sufficient for simpler applications. It > starts to show limitations, however, when we have applications with complex > logic and resultant humongous plan. The limitations are in the form of > trying to navigate the complex graph further complicated by what can fit in > your screen/viewport. > > To make it user friendly, we could try to make it interactive allowing > users to decide on what area of the plan they want focus on. There are a > couple of inspirations/examples for this. > 1. https://tensorboard.dev/experiment/rldGbR8rRHeCEbkK61SWTQ/#graphs > 2. https://cs.brown.edu/people/jcmace/d3/graph.html?id=small.json > > If we could achieve this then it will be an even more powerful resource > for spark users. > > Curious what you think > Santosh >
