Re: setting mapred.map.child.java.opts not working

2012-01-12 Thread Harsh J
Vinod, You look to be using CDH3. Lets carry this on at cdh-u...@cloudera.org [https://groups.google.com/a/cloudera.org/group/cdh-user/topics], since the issue may just be CDH3 specific. (Moving to that list, bcc'd common-user@) Could you tell us about your job and about what the per-node RAM

setting mapred.map.child.java.opts not working

2012-01-11 Thread T Vinod Gupta
Hi, Can someone help me asap? when i run my mapred job, it fails with this error - 12/01/12 02:58:36 INFO mapred.JobClient: Task Id : attempt_201112151554_0050_m_71_0, Status : FAILED Error: Java heap space attempt_201112151554_0050_m_71_0: log4j:ERROR Failed to flush writer,

Re: setting mapred.map.child.java.opts not working

2012-01-11 Thread Harsh J
These properties are not available on Apache Hadoop 1.0 (Formerly known as 0.20.x). This was a feature introduced in 0.21 (https://issues.apache.org/jira/browse/MAPREDUCE-478), and is available today on 0.22 and 0.23 line of releases. For 1.0/0.20, use mapred.child.java.opts, that applies to both

Re: setting mapred.map.child.java.opts not working

2012-01-11 Thread Koji Noguchi
Hi Harsh, Wasn't MAPREDUCE-478 in 1.0 ? Maybe the Jira is not up to date. Koji On 1/11/12 8:44 PM, Harsh J ha...@cloudera.com wrote: These properties are not available on Apache Hadoop 1.0 (Formerly known as 0.20.x). This was a feature introduced in 0.21

Re: setting mapred.map.child.java.opts not working

2012-01-11 Thread George Datskos
Koji, Harsh mapred-478 seems to be in v1, but those new settings have not yet been added to mapred-default.xml. (for backwards compatibility?) George On 2012/01/12 13:50, Koji Noguchi wrote: Hi Harsh, Wasn't MAPREDUCE-478 in 1.0 ? Maybe the Jira is not up to date. Koji On 1/11/12

Re: setting mapred.map.child.java.opts not working

2012-01-11 Thread Harsh J
@Koji - Ah, I had not checked the sources, rather just the JIRA, which I've updated now since there is indeed an svn commit listed there. Thanks for correcting me there! @George - I do not think it matters if these are present in default, but they should be present, yes. Mind filing a JIRA if

Re: setting mapred.map.child.java.opts not working

2012-01-11 Thread Koji Noguchi
but those new settings have not yet been added to mapred-default.xml. It's intentionally left out. If set in mapred-default.xml, user's mapred.child.java.opts would be ignored since mapred.{map,reduce}.child.java.opts would always win. Koji On 1/11/12 9:34 PM, George Datskos

Re: setting mapred.map.child.java.opts not working

2012-01-11 Thread T Vinod Gupta
Harsh, did you mean my job id_conf.xml? for some strange reason, i do see these 3 lines - property!--Loaded from /media/ephemeral3/hadoop/mapred/local/jobTracker/job_201201120656_0001.xml--namemapred.reduce.child.java.opts/namevalue-Xmx2048M/value/property property!--Loaded from