Shuffle Spill Memory and Shuffle Spill Disk

[Bijay Pathak]

Hello,

I am running  TeraSort <https://github.com/ehiggs/spark-terasort> on 100GB
of data. The final metrics I am getting on Shuffle Spill are:

Shuffle Spill(Memory): 122.5 GB
Shuffle Spill(Disk): 3.4 GB

What's the difference and relation between these two metrics? Does these
mean 122.5 GB was spill from memory during the shuffle?

thank you,
bijay