I see... so there really isn't a way for me to test a map/reduce program using a single node without incurring the overhead of upping/downing JVM's... My input is broken up into 5 text files.... is there a way I could start the job such that it only uses 1 map to process the whole thing? I guess I'd have to concatenate the files into 1 file and somehow turn off splitting?
Ryan On Wed, Sep 3, 2008 at 12:09 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote: > > On Sep 2, 2008, at 9:00 PM, Ryan LeCompte wrote: > >> Beginner's question: >> >> If I have a cluster with a single node that has a max of 1 map/1 >> reduce, and the job submitted has 50 maps... Then it will process only >> 1 map at a time. Does that mean that it's spawning 1 new JVM for each >> map processed? Or re-using the same JVM when a new map can be >> processed? > > It creates a new JVM for each task. Devaraj is working on > https://issues.apache.org/jira/browse/HADOOP-249 > which will allow the jvms to run multiple tasks sequentially. > > -- Owen >
