My experience with this sort of problem tells me one of two things and maybe both:
1. there are some optimizations to the code that can be made (variable re-creation inside of loops, etc.) 2. something has gone horribly wrong with the logic in the mapper. To troubleshoot I would output some log entries at specific points in the mapper (be careful not to log every execution of the mapper because this could cause major issues with the disk filling up and that sort of thing.) Hope that helps. /* Joey */ On Wed, May 23, 2012 at 2:16 PM, Mark Kerzner <mark.kerz...@shmsoft.com>wrote: > Hi, all, > > I got the exception below in the mapper. I already have my global Hadoop > heap at 5 GB, but is there a specific other setting? Or maybe I should > troubleshoot for memory? > > But the same application works in the IDE. > > Thank you! > > Mark > > *stderr logs* > > Exception in thread "Thread for syncLogs" java.lang.OutOfMemoryError: > Java heap space > at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:76) > at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:59) > at > org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:292) > at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:365) > at org.apache.hadoop.mapred.Child$3.run(Child.java:157) > Exception in thread "communication thread" java.lang.OutOfMemoryError: > Java heap space > > Exception: java.lang.OutOfMemoryError thrown from the > UncaughtExceptionHandler in thread "communication thread" >
