For several years I have been using a Hadoop 0.2 cluster successfully. I execute jobs from a remote system specifying a jar file put together on my local machine.
Suddenly all that stopped working. On some machines jobs work and on some they fail. Failures look like the text below and whem they happen I find no trace of Job201405161445 in the job tracker and no mention of the job in any entry on the slave machines. I wonder of this is a permission issue but I am running as the same user in situations where things fail and where they succeed. If there were ANY logs I might figure out the issue but I find none 14/05/26 10:12:43 INFO mapred.JobClient: Task Id : attempt_201405161445_0053_m_000101_0, Status : FAILED java.io.IOException: Task process exit with nonzero status of 1. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418) 14/05/26 10:12:44 WARN mapred.JobClient: Error reading task outputhttp:// hadoop-slave-074.ebi.ac.uk:50060/tasklog?plaintext=true&taskid=attempt_201405161445_0053_m_000101_0&filter=stdout 14/05/26 10:12:44 WARN mapred.JobClient: Error reading task outputhttp:// hadoop-slave-074.ebi.ac.uk:50060/tasklog?plaintext=true&taskid=attempt_201405161445_0053_m_000101_0&filter=stderr 14/05/26 10:12:49 INFO mapred.JobClient: Task Id : attempt_201405161445_0053_r_000003_0, Status : FAILED java.io.IOException: Task process exit with nonzero status of 1. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418) 14/05/26 10:12:49 WARN mapred.JobClient: Error reading task outputhttp:// hadoop-slave-074.ebi.ac.uk:50060/tasklog?plaintext=true&taskid=attempt_201405161445_0053_r_000003_0&filter=stdout 14/05/26 10:12:50 WARN mapred.JobClient: Error reading task outputhttp:// hadoop-slave-074.ebi.ac.uk:50060/tasklog?plaintext=true&taskid=attempt_201405161445_0053_r_000003_0&filter=stderr 14/05/26 10:12:55 INFO mapred.JobClient: Task Id : attempt_201405161445_0053_m_000101_1, Status : FAILED java.io.IOException: Task process exit with nonzero status of 1. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418)
