Re: Performing sc.paralleize (..) in workers not in the driver program

2015-06-26 Thread Akhil Das
Why do you want to do that? Thanks Best Regards On Thu, Jun 25, 2015 at 10:16 PM, shahab shahab.mok...@gmail.com wrote: Hi, Apparently, sc.paralleize (..) operation is performed in the driver program not in the workers ! Is it possible to do this in worker process for the sake of

RE: Performing sc.paralleize (..) in workers not in the driver program

2015-06-26 Thread prajod.vettiyattil
scalablity. From: Akhil Das [mailto:ak...@sigmoidanalytics.com] Sent: 26 June 2015 13:33 To: shahab Cc: user@spark.apache.org Subject: Re: Performing sc.paralleize (..) in workers not in the driver program Why do you want to do that? Thanks Best Regards On Thu, Jun 25, 2015 at 10:16 PM, shahab

RE: Performing sc.paralleize (..) in workers not in the driver program

2015-06-25 Thread Ganelin, Ilya
To: user@spark.apache.org Subject: Performing sc.paralleize (..) in workers not in the driver program Hi, Apparently, sc.paralleize (..) operation is performed in the driver program not in the workers ! Is it possible to do this in worker process for the sake of scalability? best /Shahab

Performing sc.paralleize (..) in workers not in the driver program

2015-06-25 Thread shahab
Hi, Apparently, sc.paralleize (..) operation is performed in the driver program not in the workers ! Is it possible to do this in worker process for the sake of scalability? best /Shahab