hrm, try implementing the read(DataInput in) method, as well as a
blank constructor MyWritable() that fills dummy values into your
instance variables. For example this should be all you need for
read(DataInput in),
public static MyWritable read(DataInput in) throws IOException {
MyWritable w = new MyWritable();
w.readFields(in);
return w;
}
EDIT: I was able to sort of replicate your error. In my constructor, i
had my instance variables assigned to null. Make sure you assign them
to new instances of whatever Writable you are using.
Joman Chu
http://www.notatypewriter.com/
AIM: ARcanUSNUMquam
On Sun, Oct 19, 2008 at 5:10 AM, Yih Sun Khoo <[EMAIL PROTECTED]> wrote:
> Joman to add a little bit more to one of my previous mails about the
> readFields methods
>
> Have you ever had something like this?
>
> public class MyWritable implements Writable {
> private DoubleWritable doubleWritable;
> private TupleWritable tupleWritable;
>
> public void readFields(DataInput in) throws IOException {
> doubleWritable.readFields(in);
> tupleWritable.readFields(in);
> }
>
> public void write(DataOutput out) throws IOException {
> doubleWritable.write(out);
> tupleWritable.write(out);
> }
>
>
> }
>
> On Sun, Oct 19, 2008 at 1:59 AM, Joman Chu <[EMAIL PROTECTED]> wrote:
>
>> I've never used TupleWritable, so hopefully somebody else can help you
>> with that.
>> Joman Chu
>> http://www.notatypewriter.com/
>> AIM: ARcanUSNUMquam
>>
>>
>>
>> On Sun, Oct 19, 2008 at 4:40 AM, Yih Sun Khoo <[EMAIL PROTECTED]> wrote:
>> > Also, I've noticed TupleWritable to be quite useful.
>> > What are good techniques for using TupleWritable in a mapping phase for a
>> > "list of Text" when you do not know the size of that "list" ahead of time
>> >
>> > Say I had a custom writable which implemented TupleWritable and the
>> custom
>> > writable contained a setter method
>> > mycustomwritable.setTupleWritable( ... )
>> >
>> > Where the ellipsis is, there lies the TupleWritable. However I'm
>> wondering
>> > since TupleWritable can be constructed using TupleWritable(Writable[]),
>> how
>> > do I dynamically resize the Writable[] and add Text elements to it when I
>> > don't know the size of the Writable[] very well. Does this make sense?
>> >
>> >
>> > On Sun, Oct 19, 2008 at 1:32 AM, Yih Sun Khoo <[EMAIL PROTECTED]> wrote:
>> >
>> >> Let's say in the reduce phase your value happens to hold an
>> >> ArrayListWritable
>> >> In this example, value is of type ArrayListWritable
>> >> Maybe I've not thought about this or done this before, but how does one
>> >> "read data in from the DataInput stream" in the reduce phase so that the
>> >> ArrayListWritable which is a value already passed to the reducer can be
>> used
>> >> as ArrayListWritable
>> >>
>> >>
>> >> On Sun, Oct 19, 2008 at 1:25 AM, Joman Chu <[EMAIL PROTECTED]> wrote:
>> >>
>> >>> Since the ArrayListWritable extends ArrayList, you have access to all
>> >>> the ArrayList methods as well. Once you read data in from the
>> >>> DataInput stream, you should be able to use ArrayListWritable just
>> >>> like a regular ArrayList.
>> >>> Joman Chu
>> >>> http://www.notatypewriter.com/
>> >>> AIM: ARcanUSNUMquam
>> >>>
>> >>>
>> >>>
>> >>> On Sun, Oct 19, 2008 at 4:16 AM, Yih Sun Khoo <[EMAIL PROTECTED]>
>> wrote:
>> >>> > Hmm, what method from ArrayListWritable allows you to access the
>> >>> different
>> >>> > elements of the ArrayList? Would it be readFields? for example, in
>> a
>> >>> > reduce phase, if I needed to know the size of the array list, it
>> would
>> >>> be
>> >>> > easy if i were dealing with an arraylist because i could just say
>> >>> > arraylist.size. How would i accomplish that with the writable
>> >>> counterpart?
>> >>> >
>> >>> > On Sun, Oct 19, 2008 at 1:04 AM, Joman Chu <[EMAIL PROTECTED]>
>> wrote:
>> >>> >
>> >>> >> Hi,
>> >>> >>
>> >>> >> For the ArrayList object, try taking a look at the implementation of
>> >>> >> ArrayListWritable by Jimmy Lin at UMD here:
>> >>> >>
>> >>> >>
>> >>> >>
>> >>>
>> https://subversion.umiacs.umd.edu/umd-hadoop/core/trunk/src/edu/umd/cloud9/io/ArrayListWritable.java
>> >>> >>
>> >>> >> But basically in the readFields methods, I prefer using each
>> Writable
>> >>> >> object's readFields method to read the data in. For example, for
>> your
>> >>> >> double variable, I would use a DoubleWritable object and in the
>> >>> >> MyWritable.readFields(DataInput in), I would use
>> >>> >> nameofdoublewritable.readFields(in). For the
>> >>> >> MyWritable.write(DataOutput out) method, I would use
>> >>> >> nameofdoublewritable.write(out).
>> >>> >>
>> >>> >> Have a good one,
>> >>> >>
>> >>> >> Joman Chu
>> >>> >> http://www.notatypewriter.com/
>> >>> >> AIM: ARcanUSNUMquam
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> On Sun, Oct 19, 2008 at 3:30 AM, Yih Sun Khoo <[EMAIL PROTECTED]>
>> >>> wrote:
>> >>> >> > I don't quite know how to write the read and write functions, but
>> I
>> >>> want
>> >>> >> to
>> >>> >> > write my own writable, which should have a DoubleWritable/double
>> >>> value
>> >>> >> > followed by a list of Strings/Text. This Writable will be used as
>> a
>> >>> >> value.
>> >>> >> > Is the code below the best way to go about writing such a
>> writable?
>> >>> >> >
>> >>> >> > import java.io.DataInput;
>> >>> >> > import java.io.DataOutput;
>> >>> >> > import java.io.EOFException;
>> >>> >> > import java.io.IOException;
>> >>> >> > import java.util.ArrayList;
>> >>> >> >
>> >>> >> > import org.apache.hadoop.io.Writable;
>> >>> >> >
>> >>> >> > public class MyWritable implements Writable {
>> >>> >> > private double score;
>> >>> >> > private ArrayList<String> nameList;
>> >>> >> >
>> >>> >> > public void setScore(double score) {
>> >>> >> > this.score= score;
>> >>> >> > }
>> >>> >> >
>> >>> >> > public void setNameList(ArrayList<String> nameList) {
>> >>> >> > this.nameList= nameList;
>> >>> >> > }
>> >>> >> >
>> >>> >> > public double getScore() {
>> >>> >> > return score;
>> >>> >> > }
>> >>> >> >
>> >>> >> > public ArrayList<String> getNameList() {
>> >>> >> > return nameList;
>> >>> >> > }
>> >>> >> >
>> >>> >> > public void readFields(DataInput in) throws IOException {
>> >>> >> > score= in.readDouble();
>> >>> >> > try {
>> >>> >> > do {
>> >>> >> > nameList.add(in.readUTF());
>> >>> >> > } while (true);
>> >>> >> > } catch (EOFException eofe) {
>> >>> >> > // continue; done
>> >>> >> > }
>> >>> >> > }
>> >>> >> >
>> >>> >> > public void write(DataOutput out) throws IOException {
>> >>> >> > out.writeDouble(score);
>> >>> >> > for (String name: nameList) {
>> >>> >> > out.writeUTF(name);
>> >>> >> > }
>> >>> >> > }
>> >>> >> > }
>> >>> >> >
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>
>
