Re: What executes on worker and what executes on driver side

2014-10-20 Thread Saurabh Wadhawan
What about: http://mail-archives.apache.org/mod_mbox/spark-user/201310.mbox/%3CCAF_KkPwk7iiQVD2JzOwVVhQ_U2p3bPVM=-bka18v4s-5-lp...@mail.gmail.com%3E<http://mail-archives.apache.org/mod_mbox/spark-user/201310.mbox/> Regards - Saurabh Wadhawan On 20-Oct-2014, at 4:56 pm, Kamal

Re: What executes on worker and what executes on driver side

2014-10-19 Thread Saurabh Wadhawan
join in spark : a. without broadcast(i.e. by reading a file once in each executor) b. with broadcast but by broadcasting complete RDD to each executor Regards - Saurabh Wadhawan On 19-Oct-2014, at 1:54 am, Saurabh Wadhawan mailto:saurabh.wadha...@guavus.com>> wrote: Hi,

What executes on worker and what executes on driver side

2014-10-18 Thread Saurabh Wadhawan
Hi, I have following questions: 1. When I write a spark script, how do I know what part runs on the driver side and what runs on the worker side. So lets say, I write code to to read a plain text file. Will it run on driver side only or will it run on server side only or on both sides