No idea if this is MapR specific, but looks like your mapred.child.java.opts (or map/reduce specific opts) may be incorrect for the failing jobs. Check its values up in its job.xml.
For MapR specific issues, contact MapR directly. On Thu, Sep 13, 2012 at 6:08 PM, Robin Verlangen <[email protected]> wrote: > Hi there, > > Today we started deploying Mapr M3 into production. However we're having > problems completing jobs. During a typical job the job return this: > > 12/09/11 16:33:20 INFO mapred.JobClient: Task Id : > attempt_201209111629_0002_r_000001_2, Status : FAILED on node > cl004.flxviz.com > java.lang.Throwable: Child Error > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267) > Caused by: java.io.IOException: Task process exit with nonzero status of 7. > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254) > 12/09/11 16:33:20 WARN mapred.JobClient: Error reading task output > http://cl004.flxviz.com:50060/tasklog?plaintext=true&attemptid=attempt_201209111629_0002_r_000001_2&filter=stdout > 12/09/11 16:33:20 WARN mapred.JobClient: Error reading task output > http://cl004.flxviz.com:50060/tasklog?plaintext=true&attemptid=attempt_201209111629_0002_r_000001_2&filter=stderr* > > When I get the logs of the tasktracker, I see things like: > > 2012-09-11 16:32:43,204 INFO org.apache.hadoop.mapred.TaskInProgress: Error > from attempt_201209111629_0002_r_000002_1: java.lang.Throwable: Child Error > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267) > Caused by: java.io.IOException: Task process exit with nonzero status of 7. > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254) on > tasktracker tracker_cl004.flxviz.com:localhost/127.0.0.1:53126 > 2012-09-11 16:32:46,234 INFO org.apache.hadoop.mapred.JobTracker: Removing > task 'attempt_201209111629_0002_r_000002_1' > 2012-09-11 16:32:46,512 INFO org.apache.hadoop.mapred.JobTracker: Adding > task (JOB_SETUP) 'attempt_201209111629_0002_m_000011_2' to tip > task_201209111629_0002_m_000011, for tracker > 'tracker_cl003.flxviz.com:localhost/127.0.0.1:42339' > 2012-09-11 16:32:48,027 INFO org.apache.hadoop.mapred.TaskInProgress: Error > from attempt_201209111629_0002_m_000011_2: java.lang.Throwable: Child Error > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267) > Caused by: java.io.IOException: Task process exit with nonzero status of 7. > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254) on > tasktracker tracker_cl003.flxviz.com:localhost/127.0.0.1:42339 > 2012-09-11 16:32:51,055 INFO org.apache.hadoop.mapred.JobTracker: Adding > task (JOB_SETUP) 'attempt_201209111629_0002_r_000002_2' to tip > task_201209111629_0002_r_000002, for tracker > 'tracker_cl003.flxviz.com:localhost/127.0.0.1:42339' > 2012-09-11 16:32:51,056 INFO org.apache.hadoop.mapred.JobTracker: Removing > task 'attempt_201209111629_0002_m_000011_2' > 2012-09-11 16:32:51,359 INFO org.apache.hadoop.mapred.TaskInProgress: Error > from attempt_201209111629_0002_r_000002_2: java.lang.Throwable: Child Error* > > Does anyone have a clue where to start? It doesn't seem to be a MapR > specific problem, that's why I post this in the hadoop mailinglist. > > Some additional information: > OS: Centos 6.3 x64 > 16GB Ram > 2x quad core processor > 12x 1TB harddrive > > Best regards, > > Robin Verlangen > Software engineer > > W http://www.robinverlangen.nl > E [email protected] > > Disclaimer: The information contained in this message and attachments is > intended solely for the attention and use of the named addressee and may be > confidential. If you are not the intended recipient, you are reminded that > the information remains the property of the sender. You must not use, > disclose, distribute, copy, print or rely on this e-mail. If you have > received this message in error, please contact the sender immediately and > irrevocably delete this message and any copies. > -- Harsh J
