Folks,
As Emily Latella would say, "Never mind"
I kept upping the memory for mapred.child.java.opts until the problem
went away.
Sorry for wasting bandwidth on my own stupidity.
David
On Oct 25, 2006, at 9:29 AM, David Pollak wrote:
Howdy,
I'm new to Hadoop. I've got a network of 8 machines with ~1.8TB of
storage. My first Hadoop test run is to count the URLs in a set of
crawled pages (~1.6M pages consuming about 70GB of space.) When I
run my app (or just run the Grep example) on the data set, the map
task gets to 100%, then I get an IOException and when I review the
logs, there's an OutOfMemory error listed in the tasktracker logs
("INFO org.apache.hadoop.madred.TaskRunner: task_0001_m_000258_0
java.lang.OutOfMemoryError: Java heap space.")
I've tried upping mapred.child.java.opts, but that doesn't seem to
make a difference.
Any suggestions on what I can do?
Thanks,
David
