Thanks for sharing your experience. I got the same experience -- multiple
moderate JVMs beat a single huge JVM.
Besides the minor JVM starting overhead, is it always better to have
multiple JVMs rather than a single one?
Best regards,
Wei
-
Wei Tan, PhD
Research Staff Member
IBM T. J. Watson Research Center
http://researcher.ibm.com/person/us-wtan
From: innowireless TaeYun Kim taeyun@innowireless.co.kr
To: user@spark.apache.org,
Date: 07/16/2014 05:04 AM
Subject:RE: executor-cores vs. num-executors
Thanks.
Really, now I compare a stage data of the two jobs, ‘core7-exec3’ spends
about 12.5 minutes more than ‘core2-exec12’ on GC.
From: Nishkam Ravi [mailto:nr...@cloudera.com]
Sent: Wednesday, July 16, 2014 5:28 PM
To: user@spark.apache.org
Subject: Re: executor-cores vs. num-executors
I think two small JVMs would often beat a large one due to lower GC
overhead.