mridulm commented on PR #42988: URL: https://github.com/apache/spark/pull/42988#issuecomment-1725929038
> This PR mainly focuses on the UI, independent of the profiling steps. What we might have in the future are: > > * Flame Graph Support For Task Thread Page, which [SPARK-45151](https://issues.apache.org/jira/browse/SPARK-45151) added SPARK-45151 added ability to fetch stack dump, not flame graph support - and more importantly, from point of view of flamegraphs and performance analysis, the stack dump returned suffers from safepoint bias. If this is an initial implementation, which we plan to refine over 4.0 release, that sounds fine to me - at a minimum, deployments should have the ability to override how the stack dump - for flamegraphs not SPARK-45151 - is generated. > * Add ProfilingExecutor(max, interval) message to profile the whole executor, which returns the same data structure with TriggerThreadDump message We can iterate more on this when we have a PR for it. Thanks for the details. > * Add ProfileTask(taskId, max, interval) message to profile a specific task, which returns the same data structure with TaskThreadDump message > * Different views for on/off/full CPUs > * Mixed mode profiling, which might rely upon some ext libs at runtime > * And so on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
