Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2342#issuecomment-93590771
@sarutak Thanks for the cool graphs! I haven't looked in detail into the
implementation yet, but one concern I have is scalability. It's not unheard of
in Spark to have thousands of tasks, and in this case we will render a lot of
boxes. I believe the number of items in the existing tables already makes the
page load slowly, and I would imagine that rendering these boxes will add to
that load. Have you tried viewing the UI with many many partitions? It might
make sense to display these graphs lazily.
The other thing is that for certain very short tasks, it seems to me that
the breakdown of the delays will be difficult to read. We may need to include
this breakdown in the hover menu as text.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
