Radim, You would want to modify the application master for this, and it is likely to be a bit of a hack because the RM scheduler itself is not really designed for this. You can have the AM return containers to the RM that it does not want (which it does already in some cases), because they are on the same node as an already running map task. This could slow down your application a lot, as it makes it more difficult for the RM to find free containers that the AM wants. You could also modify the AM so that it will not request any more then just one container per node, which would help with this, but not solve it completely.
--Bobby Evans On 4/27/12 12:38 AM, "Radim Kolar" <h...@filez.com> wrote: Do you want it so that the node can run only 1 mapper at a time but the node can also run multiple reducers at the same time? I currently want it per-application. Node can run more mappers/reducers from other jobs at same time. Do you want it so that this particular application will only use that node for a single mapper, but other applications may also run mappers on the node? yes. After this will be coded, i will rerun tests if it is sufficient for other types of tasks too. For start, i would need per job settings - max mappers/reducers per node, so modifying AM would be enough.
