Yes, just look at the application UI on http://<driver-node>:4040 Matei
On Nov 11, 2013, at 12:26 AM, Wenlei Xie <[email protected]> wrote: > Hi, > > I have some shuffling task which is supposed to have may repeated values, > thus I assume the shuffling compress would help the performance . > > However I get very similar running time whether I set spark.shuffle.compress > to be true/false. I would like to know whether it's because my data cannot be > compressed or not. Is there any way to monitor the data shuffled for one > transformation? > > Best, > Wenlei
