Thank you, Jeff. Actually, it turns out the scenario you outlined is much closer to the situation I'm dealing with. I'm going to ask a separate question about that.
--Jeremy On Wed, Mar 13, 2013 at 1:47 PM, Jeffrey Buell <[email protected]> wrote: > I think in your case it will have to be even, because all the slots will > get filled. A more interesting case is if you have 40 nodes, will you get > exactly 5 slots used for each of the nodes? Or will some nodes get more > than 5 mappers, and others less? I don't remember the details, but I've > had problems with unevenness in such scenarios. At least in MR1, you can > usually force evenness by adjusting the number of map and reduce slots per > node. In MR2 the slots are combined so achieving evenness will be more > difficult. > > Jeff > > ------------------------------ > *From: *"jeremy p" <[email protected]> > *To: *[email protected] > *Sent: *Wednesday, March 13, 2013 1:01:46 PM > *Subject: *Will hadoop always spread the work evenly between nodes? > > > Say I have 200 input files and 20 nodes, and each node has 10 mapper > slots. Will Hadoop always allocate the work evenly, such that each node > will get 10 input files and simultaneously start 10 mappers? Is there a > way to force this behavior? > > --Jeremy > >
