Also you'd rather have 2-3 tasks per core than 1 task per core because if the 1 task per core is actually 1.01 tasks per core, then you have one wave of tasks complete and another wave of tasks with very few tasks in them. You get better utilization when you're higher than 1.
Aaron Davidson goes into this more somewhere in this talk -- https://www.youtube.com/watch?v=dmL0N3qfSc8 On Mon, Sep 22, 2014 at 11:52 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > On Tue, Sep 23, 2014 at 1:58 AM, myasuka <myas...@live.com> wrote: > >> Thus I want to know why recommend >> 2-3 tasks per CPU core? >> > > You want at least 1 task per core so that you fully utilize the cluster's > parallelism. > > You want 2-3 tasks per core so that tasks are a bit smaller than they > would otherwise be, making them shorter and more likely to complete > successfully. > > Nick >
