The problem is that SparkPi uses Math.random(), which is a synchronized method, so it can’t scale to multiple cores. In fact it will be slower on multiple cores due to lock contention. Try another example and you’ll see better scaling. I think we’ll have to update SparkPi to create a new Random in each task to avoid this.
Matei On Apr 24, 2014, at 4:43 AM, Adnan <nsyaq...@gmail.com> wrote: > Hi, > > Relatively new on spark and have tried running SparkPi example on a > standalone 12 core three machine cluster. What I'm failing to understand is, > that running this example with a single slice gives better performance as > compared to using 12 slices. Same was the case when I was using parallelize > function. The time is scaling almost linearly with adding each slice. Please > let me know if I'm doing anything wrong. The code snippet is given below: > > > > Regards, > > Ahsan Ijaz > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/SparkPi-performance-3-cluster-standalone-mode-tp4530p4751.html > Sent from the Apache Spark User List mailing list archive at Nabble.com.