Can you please tell me , what is the functionality of those 2 methods. (How should i implement the same in this VectorWritable) ..
Thanks On Tue, Dec 29, 2009 at 11:25 AM, Jeff Zhang <[email protected]> wrote: > The readFields and write method is empty ? > > When data is transfered from map phase to reduce phase, data is serialized > and deserialized , so the write and readFields will be called. You should > not leave them empty. > > > Jeff Zhang > > > On Tue, Dec 29, 2009 at 1:29 PM, bharath v < > [email protected]> wrote: > > > Hi , > > > > I've implemented a simple VectorWritable class as follows > > > > > > package com; > > > > import org.apache.hadoop.*; > > import org.apache.hadoop.io.*; > > import java.io.*; > > import java.util.Vector; > > > > > > public class VectorWritable implements WritableComparable { > > private Vector<String> value = new Vector(); > > > > public VectorWritable() {} > > > > public VectorWritable(Vector<String> value) { set(value); } > > > > public void set(Vector<String> val) { this.value = val; > > } > > > > public Vector<String> get() { return this.value; } > > > > public void readFields(DataInput in) throws IOException { > > //value = in.readInt(); > > } > > > > public void write(DataOutput out) throws IOException { > > // out.writeInt(value); > > } > > > > public boolean equals(Object o) { > > if (!(o instanceof VectorWritable)) > > return false; > > VectorWritable other = (VectorWritable)o; > > return this.value.equals(other.value); > > } > > > > public int hashCode() { > > return value.hashCode(); > > } > > > > public int compareTo(Object o) { > > Vector thisValue = this.value; > > Vector thatValue = ((VectorWritable)o).value; > > return (thisValue.size()<thatValue.size() ? -1 : > > (thisValue.size()==thatValue.size() ? 0 : 1)); > > } > > > > public String toString() { > > return value.toString(); > > } > > > > public static class Comparator extends WritableComparator { > > public Comparator() { > > super(VectorWritable.class); > > } > > > > public int compare(byte[] b1, int s1, int l1, > > byte[] b2, int s2, int l2) { > > > > int thisValue = readInt(b1, s1); > > int thatValue = readInt(b2, s2); > > return (thisValue<thatValue ? -1 : (thisValue==thatValue ? 0 : 1)); > > } > > } > > > > static { // register this > > comparator > > WritableComparator.define(VectorWritable.class, new Comparator()); > > } > > } > > > > The map phase is outputting correct <Text,VectorWritable> pairs .. but in > > reduce phase > > when I iterate over the values Iterable.. Iam getting the size of the > > vector > > to be 0; I think there is a minor > > mistake in my VectorWritable Implementation .. Can anyone point it.. > > > > Thanks > > >
