The readFields and write method is empty ? When data is transfered from map phase to reduce phase, data is serialized and deserialized , so the write and readFields will be called. You should not leave them empty.
Jeff Zhang On Tue, Dec 29, 2009 at 1:29 PM, bharath v < [email protected]> wrote: > Hi , > > I've implemented a simple VectorWritable class as follows > > > package com; > > import org.apache.hadoop.*; > import org.apache.hadoop.io.*; > import java.io.*; > import java.util.Vector; > > > public class VectorWritable implements WritableComparable { > private Vector<String> value = new Vector(); > > public VectorWritable() {} > > public VectorWritable(Vector<String> value) { set(value); } > > public void set(Vector<String> val) { this.value = val; > } > > public Vector<String> get() { return this.value; } > > public void readFields(DataInput in) throws IOException { > //value = in.readInt(); > } > > public void write(DataOutput out) throws IOException { > // out.writeInt(value); > } > > public boolean equals(Object o) { > if (!(o instanceof VectorWritable)) > return false; > VectorWritable other = (VectorWritable)o; > return this.value.equals(other.value); > } > > public int hashCode() { > return value.hashCode(); > } > > public int compareTo(Object o) { > Vector thisValue = this.value; > Vector thatValue = ((VectorWritable)o).value; > return (thisValue.size()<thatValue.size() ? -1 : > (thisValue.size()==thatValue.size() ? 0 : 1)); > } > > public String toString() { > return value.toString(); > } > > public static class Comparator extends WritableComparator { > public Comparator() { > super(VectorWritable.class); > } > > public int compare(byte[] b1, int s1, int l1, > byte[] b2, int s2, int l2) { > > int thisValue = readInt(b1, s1); > int thatValue = readInt(b2, s2); > return (thisValue<thatValue ? -1 : (thisValue==thatValue ? 0 : 1)); > } > } > > static { // register this > comparator > WritableComparator.define(VectorWritable.class, new Comparator()); > } > } > > The map phase is outputting correct <Text,VectorWritable> pairs .. but in > reduce phase > when I iterate over the values Iterable.. Iam getting the size of the > vector > to be 0; I think there is a minor > mistake in my VectorWritable Implementation .. Can anyone point it.. > > Thanks >
