Yep that was it. I changed the method signature to LongWriteable for
the 1st parameter (also annotated it w/ Override to see the compile
error)
thanks for your help!
On Oct 15, 2009, at 6:40 PM, Ahad Rana wrote:
Hi,
Unfortunately Mapper is now a class and from your call stack it
seems that
Mapper's default map implementation is being called (instead of the
one you
defined in your class), which is passing the LongWritable key to the
collector. You should use @Override to have the compiler help you
figure out
why your map function doesn't have the same signature as the one
defined in
the base class.
Best of luck.
Ahad.
On Thu, Oct 15, 2009 at 10:29 AM, yz5od2 <woods5242-
[email protected]>wrote:
Hi,
I am new to Hadoop so this might be an easy question for someone to
help me
with.
I continually am getting this exception (my code follows below)
java.io.IOException: Type mismatch in key from map: expected
org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable
at
org.apache.hadoop.mapred.MapTask
$MapOutputBuffer.collect(MapTask.java:807)
at
org.apache.hadoop.mapred.MapTask
$NewOutputCollector.write(MapTask.java:504)
at
org
.apache
.hadoop
.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:
80)
at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:
583)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Running 0.20.1
I have a text file with lines of data separated by carriage
returns. This
is properly stored in a directory within HDFS. I only have a
Mapping task
for processing this file, after the mapping is done it should go
straight to
output, No reduce or combiner functions.
I am just trying to test to see if this will run. The mapper just
takes the
data sent to it and adds it back to the collector as 2 text values.
--------------------
My Mapper:
--------------------
public class MyTestMapper extends
Mapper<LongWritable,Text,Text,Text> {
public void map(Object key, Text value, Context context) throws
IOException, InterruptedException {
String key = key.toString();
String line = value.toString();
String id = extractId(line);
String reformattedLine = reformatLine(line);
context.write(new Text(id), new Text(reformattedLine));
}
}
--------------------
My job submission code:
-------------------------------------
Job job = new Job(conf);
job.setJarByClass(MyTestMapper.class);
job.setMapperClass(MyTestMapper.class);
FileInputFormat.addInputPath(job, new Path("/myDir/sample.txt"));
FileOutputFormat.setOutputPath(job, new
Path("/myDir/output/results-"+System.currentTimeMillis()+".txt"));
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
job.submit();
