[ http://issues.apache.org/jira/browse/HADOOP-794?page=all ]
Owen O'Malley updated HADOOP-794:
---------------------------------
Attachment: div-zero.patch
This patch protects against requesting 0 maps. The sort benchmark still will
have problems if it is run with 0 task trackers because it will request 0
reduces. The proper fix for tat is to wait until there are a reasonable number
of task trackers before submitting the job. This patch also adds a test case
that requests 0 maps.
> JobTracker crashes with ArithmeticException
> -------------------------------------------
>
> Key: HADOOP-794
> URL: http://issues.apache.org/jira/browse/HADOOP-794
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Nigel Daley
> Assigned To: Owen O'Malley
> Fix For: 0.10.0
>
> Attachments: div-zero.patch
>
>
> The sort benchmark on 20 nodes failed for me with an ArithmeticException.
> Sort.java calculates the number of maps and reduces dynamically based on the
> cluster size. I'm guessing at the time it did the calculation, the
> jobtracker had heard from 0 task trackers (which seems odd since I wait 1
> minute after starting the jobtracker before starting Sort).
> From the job output:
> Running on 0 nodes to sort from /sortBenchmark20/input into
> /sortBenchmark20/output with 0 reduces.
> From the JobTracker log:
> 2006-12-07 07:10:06,709 ERROR org.apache.hadoop.mapred.JobTracker: Job
> initialization failed:
> java.lang.ArithmeticException: / by zero
> at
> org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:130)
> at
> org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:125)
> at
> org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:326)
> at java.lang.Thread.run(Thread.java:595)
> The jobtacker should check for illegal inputs like this and not fall over.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
