Hi, I have some shuffling task which is supposed to have may repeated values, thus I assume the shuffling compress would help the performance .
However I get very similar running time whether I set spark.shuffle.compress to be true/false. I would like to know whether it's because my data cannot be compressed or not. Is there any way to monitor the data shuffled for one transformation? Best, Wenlei
