Yes, I see the same thing. Adding maxmemory="128m" to the junit ant task fixed it for me. Alternatively we may be able to turn down some of the Hadoop memory usage parameters such as sort and in memory fs space.
I see some other errors: RAT fails, with: Unapproved licenses: /Users/scottc/Documents/workspace/avro-trunk/build/avro-src-1.4.0-SNAPSHOT/lang/ruby/test/test_datafile.rb Java unit tests fail mysteriously for me after TestUtf8 -- no errors are logged and no tests report a failure in the output. It might be an environment issue on my machine however. On Apr 3, 2010, at 6:09 PM, Jeff Hodges wrote: > Anyone else seeing OutOfMemory errors when trying to run the hadoop tests? > > Specifically: > > > [junit] Running org.apache.avro.mapred.TestWordCountGeneric > [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.metrics.jvm.JvmMetrics > init > [junit] INFO: Initializing JVM Metrics with > processName=JobTracker, sessionId= > [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.mapred.JobClient > configureCommandLineOptions > [junit] WARNING: Use GenericOptionsParser for parsing the > arguments. Applications should implement Tool for the same. > [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient > configureCommandLineOptions > [junit] WARNING: No job jar file set. User classes may not be > found. See JobConf(Class) or JobConf#setJar(String). > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.FileInputFormat listStatus > [junit] INFO: Total input paths to process : 1 > [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: Running job: job_local_0001 > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.FileInputFormat listStatus > [junit] INFO: Total input paths to process : 1 > [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.MapTask > runOldMapper > [junit] INFO: numReduceTasks: 1 > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> > [junit] INFO: io.sort.mb = 100 > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.LocalJobRunner$Job run > [junit] WARNING: job_local_0001 > [junit] java.lang.OutOfMemoryError: Java heap space > [junit] at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781) > [junit] at > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350) > [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > [junit] at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: map 0% reduce 0% > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: Job complete: job_local_0001 > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.Counters log > [junit] INFO: Counters: 0 > [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.29 sec > [junit] Error: > [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.metrics.jvm.JvmMetrics > init > [junit] INFO: Initializing JVM Metrics with > processName=JobTracker, sessionId= > [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.mapred.JobClient > configureCommandLineOptions > [junit] WARNING: Use GenericOptionsParser for parsing the > arguments. Applications should implement Tool for the same. > [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient > configureCommandLineOptions > [junit] WARNING: No job jar file set. User classes may not be > found. See JobConf(Class) or JobConf#setJar(String). > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.FileInputFormat listStatus > [junit] INFO: Total input paths to process : 1 > [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: Running job: job_local_0001 > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.FileInputFormat listStatus > [junit] INFO: Total input paths to process : 1 > [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.MapTask > runOldMapper > [junit] INFO: numReduceTasks: 1 > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> > [junit] INFO: io.sort.mb = 100 > [junit] Apr 3, 2010 5:57:57 PM > org.apache.hadoop.mapred.LocalJobRunner$Job run > [junit] WARNING: job_local_0001 > [junit] java.lang.OutOfMemoryError: Java heap space > [junit] at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781) > [junit] at > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350) > [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > [junit] at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: map 0% reduce 0% > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: Job complete: job_local_0001 > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.Counters log > [junit] INFO: Counters: 0 > [junit] > [junit] TEST org.apache.avro.mapred.TestWordCountGeneric FAILED > [junit] Running org.apache.avro.mapred.TestWordCountSpecific > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.metrics.jvm.JvmMetrics > init > [junit] INFO: Cannot initialize JVM Metrics with > processName=JobTracker, sessionId= - already initialized > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient > configureCommandLineOptions > [junit] WARNING: Use GenericOptionsParser for parsing the > arguments. Applications should implement Tool for the same. > [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient > configureCommandLineOptions > [junit] WARNING: No job jar file set. User classes may not be > found. See JobConf(Class) or JobConf#setJar(String). > [junit] Apr 3, 2010 5:57:58 PM > org.apache.hadoop.mapred.FileInputFormat listStatus > [junit] INFO: Total input paths to process : 1 > [junit] Apr 3, 2010 5:57:59 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: Running job: job_local_0002 > [junit] Apr 3, 2010 5:57:59 PM > org.apache.hadoop.mapred.FileInputFormat listStatus > [junit] INFO: Total input paths to process : 1 > [junit] Apr 3, 2010 5:57:59 PM org.apache.hadoop.mapred.MapTask > runOldMapper > [junit] INFO: numReduceTasks: 1 > [junit] Apr 3, 2010 5:57:59 PM > org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> > [junit] INFO: io.sort.mb = 100 > [junit] Apr 3, 2010 5:57:59 PM > org.apache.hadoop.mapred.LocalJobRunner$Job run > [junit] WARNING: job_local_0002 > [junit] java.lang.OutOfMemoryError: Java heap space > [junit] at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781) > [junit] at > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350) > [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > [junit] at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) > [junit] Apr 3, 2010 5:58:00 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: map 0% reduce 0% > [junit] Apr 3, 2010 5:58:00 PM org.apache.hadoop.mapred.JobClient > monitorAndPrintJob > [junit] INFO: Job complete: job_local_0002 > [junit] Apr 3, 2010 5:58:00 PM org.apache.hadoop.mapred.Counters log > [junit] INFO: Counters: 0 > [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 1.58 sec > [junit] TEST org.apache.avro.mapred.TestWordCountSpecific FAILED
