Hi, I was tryin to run a map reduce program which reads a txt file filled wid some keywords, my map task takes each of these keywords, does some processing and returns a complex object url which contains media, status, link and title (each being a string). my reduce class simply has one line while(value.hasnext()){ output.collect(key, value.next()) } i tried the following configuration: conf.setInputFormat(TextInputFormat.class); conf.setOutputFormat(TextOutputFormat.class); conf.setMapOutputKeyClass(Text.class); conf.setMapOutputValueClass(PObjectWritable.class); conf.setOutputKeyClass(Text.class); conf.setOutputValueClass(PObjectWritable.class);
Following is my PObjectWritable object: ************************************************************************************************************************************************************************ import java.io.DataInput; import java.io.DataOutput; import java.io.IOException; import org.apache.hadoop.io.Writable; public class PObjectWritable implements Writable{ private String url; private String media; private String title; private String status; public String getUrl() { return url; } public void setUrl(String url) { this.url = url; } public String getMedia() { return media; } public void setMedia(String media) { this.media = media; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } public String getStatus() { return status; } public void setStatus(String status) { this.status = status; } @Override public void readFields(DataInput in) throws IOException { // TODO Auto-generated method stub if(in.readBoolean()){ url = in.readUTF(); }else{ url = null; } if(in.readBoolean()){ media = in.readUTF(); }else{ media = null; } if(in.readBoolean()){ title = in.readUTF(); }else{ title = null; } if(in.readBoolean()){ status = in.readUTF(); }else{ status = null; } } @Override public void write(DataOutput out) throws IOException { // TODO Auto-generated method stub if(url!=null){ out.writeUTF(url); } if(media!=null){ out.writeUTF(media); } if(title!=null){ out.writeUTF(title); } if(status!=null){ out.writeUTF(status); } } @Override public String toString() { // TODO Auto-generated method stub return url + ", " + media + ", " + title + ", " + status; } } ************************************************************************************************************************************************************************** Following is the exception at reduce: java.lang.RuntimeException: problem advancing post rec#0 at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:831) at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237) at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233) at org.apache.hadoop.mapred.lib.IdentityReducer.reduce(IdentityReducer.java:39) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:430) at org.apache.hadoop.mapred.Child.main(Child.java:155) Caused by: java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:180) at java.io.DataInputStream.readUTF(DataInputStream.java:592) at java.io.DataInputStream.readUTF(DataInputStream.java:547) at com.mapreduce.PObjectWritable.readFields(PObjectWritable.java:59) at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67) at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40) at org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:888) at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:828) ... 5 more wht could cause the exception?? Regards Raakhi