All else equal it is better to have the same resources in fewer executors. More tasks are local to other tasks which helps perf. There is more possibility of 'borrowing' extra mem and CPU in a task.
On Thu, Mar 16, 2023, 2:14 PM Nikhil Goyal <nownik...@gmail.com> wrote: > Hi folks, > I am trying to understand what would be the difference in running 8G 1 > core executor vs 40G 5 core executors. I see that on yarn it can cause bin > fitting issues but other than that are there any pros and cons on using > either? > > Thanks > Nikhil >
