Re: is there anyway to enforce Spark to cache data in all worker nodes(almost equally) ?

2015-04-30 Thread shahab
Thanks Alex, but 482MB was just example size, and I am looking for generic approach doing this without broadcasting, any idea? best, /Shahab On Thu, Apr 30, 2015 at 4:21 PM, Alex wrote: > 482 MB should be small enough to be distributed as a set of broadcast > variables. Then you can use loca

RE: is there anyway to enforce Spark to cache data in all worker nodes(almost equally) ?

2015-04-30 Thread Alex
482 MB should be small enough to be distributed as a set of broadcast variables. Then you can use local features of spark to process. -Original Message- From: "shahab" Sent: ‎4/‎30/‎2015 9:42 AM To: "user@spark.apache.org" Subject: is there anyway to enforce Spark to cache data in all w