Have a look at org.apache.hadoop.io.ArrayWritable. You may be able to use this class in your application, or at least use it as a basis for writing VectorWritable.
Cheers, Tom On Tue, Dec 29, 2009 at 1:37 AM, bharath v <[email protected]> wrote: > Can you please tell me , what is the functionality of those 2 methods. > (How should i implement the same in this VectorWritable) .. > > Thanks > > On Tue, Dec 29, 2009 at 11:25 AM, Jeff Zhang <[email protected]> wrote: > >> The readFields and write method is empty ? >> >> When data is transfered from map phase to reduce phase, data is serialized >> and deserialized , so the write and readFields will be called. You should >> not leave them empty. >> >> >> Jeff Zhang >> >> >> On Tue, Dec 29, 2009 at 1:29 PM, bharath v < >> [email protected]> wrote: >> >> > Hi , >> > >> > I've implemented a simple VectorWritable class as follows >> > >> > >> > package com; >> > >> > import org.apache.hadoop.*; >> > import org.apache.hadoop.io.*; >> > import java.io.*; >> > import java.util.Vector; >> > >> > >> > public class VectorWritable implements WritableComparable { >> > private Vector<String> value = new Vector(); >> > >> > public VectorWritable() {} >> > >> > public VectorWritable(Vector<String> value) { set(value); } >> > >> > public void set(Vector<String> val) { this.value = val; >> > } >> > >> > public Vector<String> get() { return this.value; } >> > >> > public void readFields(DataInput in) throws IOException { >> > //value = in.readInt(); >> > } >> > >> > public void write(DataOutput out) throws IOException { >> > // out.writeInt(value); >> > } >> > >> > public boolean equals(Object o) { >> > if (!(o instanceof VectorWritable)) >> > return false; >> > VectorWritable other = (VectorWritable)o; >> > return this.value.equals(other.value); >> > } >> > >> > public int hashCode() { >> > return value.hashCode(); >> > } >> > >> > public int compareTo(Object o) { >> > Vector thisValue = this.value; >> > Vector thatValue = ((VectorWritable)o).value; >> > return (thisValue.size()<thatValue.size() ? -1 : >> > (thisValue.size()==thatValue.size() ? 0 : 1)); >> > } >> > >> > public String toString() { >> > return value.toString(); >> > } >> > >> > public static class Comparator extends WritableComparator { >> > public Comparator() { >> > super(VectorWritable.class); >> > } >> > >> > public int compare(byte[] b1, int s1, int l1, >> > byte[] b2, int s2, int l2) { >> > >> > int thisValue = readInt(b1, s1); >> > int thatValue = readInt(b2, s2); >> > return (thisValue<thatValue ? -1 : (thisValue==thatValue ? 0 : 1)); >> > } >> > } >> > >> > static { // register this >> > comparator >> > WritableComparator.define(VectorWritable.class, new Comparator()); >> > } >> > } >> > >> > The map phase is outputting correct <Text,VectorWritable> pairs .. but in >> > reduce phase >> > when I iterate over the values Iterable.. Iam getting the size of the >> > vector >> > to be 0; I think there is a minor >> > mistake in my VectorWritable Implementation .. Can anyone point it.. >> > >> > Thanks >> > >> >
