Ah, thank you. Eric's response really cleared that up. So it's the iterator that is re-using the same Text object over and over.
About making the deep copy, Eric writes "If you really need to build a list like this, you'd have to resort to doing a deep copy, but you're better off avoid it if you can as it will drastically impact performance". Would it be more efficient to simply copy the byte array returned from getBytes() to do the comparison? -Steven Willis -----Original Message----- From: Harsh J [mailto:ha...@cloudera.com] Sent: Friday, March 16, 2012 3:32 AM To: mapreduce-user@hadoop.apache.org Subject: Re: Text Iterable modifying other variable What Brock said. Also read Eric Sammers' response not too long ago on http://search-hadoop.com/m/Ok1JT1YABHb1 for some more info. On Thu, Mar 15, 2012 at 6:00 AM, Brock Noland <br...@cloudera.com> wrote: > Writable's are reused. You need to deep copy the "value" into "lastValue". > > On Wed, Mar 14, 2012 at 6:49 PM, Steven Willis <swil...@compete.com> wrote: >> I seem to have made a few typos in the code. This is how it should read: >> >> /*****************/ >> public class MyReducer extends Reducer<Text, Text, Text, Text> { >> @Override >> public void reduce(Text key, Iterable<Text> values, Reducer<Text, Text, >> Text, Text>.Context context) throws IOException, InterruptedException { >> Text value = new Text(); >> Text lastValue = new Text(); >> Iterator<Text> valuesIterator = values.iterator(); >> >> while(valuesIterator.hasNext()) { >> value = valuesIterator.next(); >> if(!value.equals(lastValue)){ >> context.write(key, value); >> lastValue = value; >> } >> } >> } >> } >> /*****************/ >> >> My description of the problem stands. >> >> -Steven Willis > > > > -- > Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/ -- Harsh J
