We have jobs that require different resources and as such saturate our
machines at different levels or parallelization.
What we want to do in the driver is set the number of simultaneous jobs
per node.
JobClient client = new JobClient();
Configuration configuration = new Configuration();
configuration.setInt( "mapred.tasktracker.tasks.maximum", 7);
JobConf conf = new JobConf(configuration,MergeNewSeenDriver.class);
System.err.println( "configured maximum tasks is " + conf.get(
"mapred.tasktracker.tasks.maximum" ));
But this doesn't seem to work. The only success we have had is using
multithreaded map runner, but then we don't get to run multiple reduces
at a time on the machines.
Any suggestions?
