>From 2 to 4, the performance increase sub-linearly, however from 4 to 8, it seems super-linear.
Is it caused by some disk contention bottleneck? 2013/9/6 牛兆捷 <nzjem...@gmail.com> > Hi all： > > I vary the computational nodes of cluster and get the speedup result in > attachment. > > In my mind, there are three type of speedup model: linear, sub-linear and > super-linear. However the curve of my result seems a little strange. I have > attached it. > [image: 内嵌图片 2] > > This is sort in example.jar, actually it is done only using the default > map-reduce mechanism of Hadoop. > > I use hadoop-1.2.1, set 8 map slots and 8 reduce slots per node(12 cpu, > 20g men) > io.sort.mb = 512, block size = 512mb, heap size = 1024mb, > reduce.slowstart = 0.05, the others are default. > > Input data: 20g, I divide it to 64 files > > Sort example: 64 map tasks, 64 reduce tasks > > Computational nodes: varying from 2 to 9 > > Why the speedup mechanism is like this? How can I model it properly? > > Thanks～ > > -- > *Sincerely,* > *Zhaojie* > * > * > -- *Sincerely,* *Zhaojie* * *
