InputSampler does not create a deep copy of the key class when creating a
sample, which causes problems with some formats like SequenceFile<Text,Text>
------------------------------------------------------------------------------------------------------------------------------------------------------
Key: MAPREDUCE-1820
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1820
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Alex Kozlov
I tried to use the InputSampler on a SequenceFile<Text,Text> and found that it
comes up with duplicate keys in the sample. The problem was tracked down to
the fact that the Text object returned from the reader is essentially a wrapper
pointing to a byte array, which changes as the sequence file reader progresses.
There was also a bug in that the reader should be initialized before the use.
The am attaching a patch that fixes both of the issues. --Alex K
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
