Hi All, I have a machine with the following configuration: 32 GB RAM 500 GB HDD 8 CPUs
Following are the parameters i'm starting my Spark context with: val conf = new SparkConf().setAppName("MasterApp").setMaster("local[1]").set("spark.executor.memory", "20g") I'm reading a 4.3 GB file and counting the no. of characters in it. When i run my program with: local[1], the count is returned in 1.8 minutes local[8], the count is returned in 4.2 minutes Both the times, Spark uses Storage Memory of 3.7 GB. Why would my program take more time with local[8]? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-master-takes-more-time-with-local-8-than-local-1-tp26052.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org