Hi All, I'm wonderring why does shuffle in spark write shuffle data to disk by default? In Stackoverflow, someone said it's used by FTS, but node down is the most common reason of fault, and write to disk cannot do FTS in this case either. So why not use ramdisk as default instread of SDD or HDD only?
Thanks Hubert Zhang