Spill can fail with bad call to Random
--------------------------------------
Key: HADOOP-6766
URL: https://issues.apache.org/jira/browse/HADOOP-6766
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 0.20.2
Reporter: Peter Arthur Ciccolo
Priority: Minor
java.lang.IllegalArgumentException: n must be positive
at java.util.Random.nextInt(Random.java:250)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:243)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:289)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
at
org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1221)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1129)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:549)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:623)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
confChanged assumes that the list of dirs it creates
(LocalDirAllocator.java:215) has at least one element in it by the end of the
function. If, for each local dir, either the conditional on line 221 is false,
or the call to DiskChecker.checkDir() throws an exception, this assumption will
not hold. In this case, dirIndexRandomizer.nextInt() is called on the number of
elements in dirs, which is 0. Since dirIndexRandomizer (195) is an instance of
Random(), it needs a positive (non-zero) argument to nextInt().
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
