Hello, We wanted to tune the Spark running on YARN cluster.The Spark History Server UI shows lots of parameters like:
- GC time - Task Duration - Shuffle R/W - Shuffle Spill (Memory/Disk) - Serialization Time (Task/Result) - Scheduler Delay Among the above metrics, which are the most important that should be taken as reference for benchmarking the cluster performance? Thanks, Bijay