I created my own Writable class to store 3 pieces of information. In
my mapreducer.Reducer class I collect all of them and then process as
a group, ie:
reduce(key, values, context) {
List<Foo> myFoos =new ArrayList();
for (Foo value : values) {
myFoos.add(value);
}
}
I was perplexed when entries in the list changed underneath me so I
put a freeze() method on it:
boolean m_frozen = false;
public void freeze() {
m_frozen = true;
}
@Override
public void readFields(DataInput in) throws IOException {
if (m_frozen) {
throw new IllegalStateException();
}
}
And noted that the exception was thrown:
at Foo.readFields(Foo.java:169)
at org.apache.hadoop.io.serializer.WritableSerialization
$WritableDeserializer.deserialize(WritableSerialization.java:67)
at org.apache.hadoop.io.serializer.WritableSerialization
$WritableDeserializer.deserialize(WritableSerialization.java:40)
at
org
.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext.java:
116)
at org.apache.hadoop.mapreduce.ReduceContext
$ValueIterator.next(ReduceContext.java:163)
Am I doing something wrong? Should I expect this VALUEIN object to
change from underneath me? I'm using hadoop 0.20.1 (from a cloudera
tarball)
Chris