Hi Christian, Generally along with a nonzero exit code you should see something in the stderr for that attempt. If you look on the TaskTracker inside logs/userlogs/attempt_<the failed attempt>/stderr do you see anything useful?
If it's a segfault or a linux OOM kill, you should also see something in your system's kernel log. Check "dmesg" and/or /var/log/kern.log for anything suspicious looking. Hope that helps -Todd On Tue, Jul 21, 2009 at 2:15 AM, Christian Kirschbaum < [email protected]> wrote: > Hi all, > > we're using Hadoop 0.19.1 and have recently encountered the following > erratic problem when running jobs involving UIMA text annotation chains > (which fail frequently because of this): > > java.io.IOException: Task process exit with nonzero status of 134. > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:425) > > > As you can see, this is propagated in Hadoop code, without the actual > MapReduce job being able to react to it. Unfortunately, this exception > message isn't very descriptive as to the actual cause which I have yet to > track down. > > All I found out is that this status code apparently is an exit code of a > separate process initiated through > org.apache.hadoop.util.Shell.ShellCommandExecutor in the runChild(JvmEnv) > method of org.apache.hadoop.mapred.JvmManager. And because it is exit code > 134 (128 + 6), supposedly signal 6 (ABORT) has effected the process > termination which may indicate a core dump? > > How do I find out more about the actual cause? Is there any secret logfile > for the separately spawned Jvm process? I've looked through various logs and > userlogs directories but could not find any mention of this exception there. > > Any help is appreciated. > > Thanks, > Chris > > -- > Christian Kirschbaum > Software Developer > -------------------------------------------------------- > vionto GmbH > Karl-Marx-Allee 90a, D-10243 Berlin > > fon +49 30 40 20 329 - 27 > fax +49 30 40 20 329 - 01 > web http://www.vionto.com > -------------------------------------------------------- > Geschäftsführer: Ralf von Grafenstein, Dr. Martin Hirsch > Sitz der Gesellschaft: Berlin > Amtsgericht Berlin Charlottenburg, HRB 108154B > -------------------------------------------------------- > >
