Does anyone know if this is the right way to configure Hadoop from a Crunch DoFn? This didn't seem to affect anything.
Thanks!
@Override
public void configure(Configuration conf) {
conf.set("mapreduce.map.java.opts", "-Xmx3900m");
conf.set("mapreduce.reduce.java.opts", "-Xmx3900m");
conf.set("mapreduce.map.memory.mb", "4096");
conf.set("mapreduce.reduce.memory.mb", "4096");
}
