Re: reasonable number of executors

Hi Igor, That's a great talk and an exact answer to my question. Thank you. Cheers, Alex. On Tue, Feb 23, 2016 at 8:27 PM, Igor Berman wrote: > > http://www.slideshare.net/cloudera/top-5-mistakes-to-avoid-when-writing-apache-spark-applications > > there is a section that is connected to your q

Re: reasonable number of executors

http://www.slideshare.net/cloudera/top-5-mistakes-to-avoid-when-writing-apache-spark-applications there is a section that is connected to your question On 23 February 2016 at 16:49, Alex Dzhagriev wrote: > Hello all, > > Can someone please advise me on the pros and cons on how to allocate the >

Re: reasonable number of executors

Hi Alex, take a look here : https://blogs.aws.amazon.com/bigdata/post/Tx3RD6EISZGHQ1C/The-Impact-of-Using-Latest-Generation-Instances-for-Your-Amazon-EMR-Job Bas

reasonable number of executors

Hello all, Can someone please advise me on the pros and cons on how to allocate the resources: many small heap machines with 1 core or few machines with big heaps and many cores? I'm sure that depends on the data flow and there is no best practise solution. E.g. with bigger heap I can perform map-