Hi Todd, Are these directories supposed to be on the namenode or on each of the datanodes ? In my case it is set to a directory inside /tmp but the mapred.local.dir was present only on the namenode.
Thanks for the help Himanshu Morpheus: Do you believe in fate, Neo? Neo: No. Morpheus: Why Not? Neo: Because I don't like the idea that I'm not in control of my life. ----- Original Message ---- From: Todd Lipcon <[email protected]> To: [email protected] Sent: Thu, December 31, 2009 10:17:05 AM Subject: Re: large reducer output with same key Hi Himanshu, Sounds like your mapred.local.dir doesn't have enough space. My guess is that you've configured it somewhere inside /tmp/. Instead you should spread it across all of your local physical disks by comma-separating the directories in the configuration. Something like: <property> <name>mapred.local.dir</name> <value>/disk1/mapred-local,/disk2/mapred-local,/disk3/mapred-local</value> </property> (and of course make sure those directories exist and are writable by the user that runs your hadoop daemons, often "hadoop") Thanks -Todd On Thu, Dec 31, 2009 at 2:10 AM, himanshu chandola < [email protected]> wrote: > Hi Everyone, > My reducer output results in most of the data having the same key. The > reducer output is close to 16 GB and though my cluster in total has a > terabyte of space in hdfs I get errors like the following : > > > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:719) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:209) > > at > > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084) > > Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: > > Could not find any valid local directory for > > task_200808021906_0002_m_000014_2/spill4.out > > After such failures, hadoop tries to start the same reduce job couple times > on other nodes before the job fails. From the > exception, it looks to me this is > probably a disk error(some machines have less than 16 gigs free space on > hdfs). > > So my question was whether hadoop puts values which share the same key as a > single block in one node ? Or something else > could be happening here ? > > Thanks > > H > > > >
