Yes you're right, but I haven't understand what you said.
- Why this should be happening? Is this related to the
mapred.mapper.new-api property?
- Why the function signature doesn't match? I mean, what's the reason
that this should be happening? I don't recall to change the code that
could give this error.
On Thu, Jan 27, 2011 at 4:29 PM, Pedro Costa <psdc1...@gmail.com> wrote:
> Yes, that's the one that's being used ( o.a.h.mapreduce.Mapper ). This
> is not the right one to use?
>
>
>
> On Thu, Jan 27, 2011 at 3:40 PM, Chase Bradford
> <chase.bradf...@gmail.com> wrote:
>> Are you sure the function signature for you Mapper's map matches the super
>> class, and that you specified your Map class in the job setup? It sounds a
>> bit like the base o.a.h.mapreduce.Mapper map implementation is being used
>> instead.
>>
>>
>> On Jan 27, 2011, at 2:36 AM, Pedro Costa <psdc1...@gmail.com> wrote:
>>
>>> The map output class are well defined:
>>> keyClass: class org.apache.hadoop.io.BooleanWritable - valClass: class
>>> org.apache.hadoop.io.LongWritable
>>>
>>> but executing the pi example, the values that map function passes is:
>>> keyClass: class org.apache.hadoop.io.LongWritable - valClass: class
>>> org.apache.hadoop.io.Text
>>>
>>>
>>> I looked at the PiEstimator.class.PiMapper#map function, and the
>>> output collector seems ok.
>>>
>>> [code]
>>> public void map(LongWritable offset,
>>> LongWritable size,
>>> OutputCollector<BooleanWritable, LongWritable> out,
>>> Reporter reporter) throws IOException {
>>> (...)
>>> out.collect(new BooleanWritable(true), new LongWritable(numInside));
>>> out.collect(new BooleanWritable(false), new LongWritable(numOutside));
>>> }
>>> [/code]
>>>
>>> I'm really confused, right now. How can this be happening?
>>>
>>>
>>> On Thu, Jan 27, 2011 at 10:19 AM, Pedro Costa <psdc1...@gmail.com> wrote:
>>>> Thanks Nicholas, but it didn't worked.
>>>>
>>>> Can I do a remote debugging on hadoop examples? I really like to put a
>>>> breakpoint in the Pi class.
>>>>
>>>> Thanks,
>>>>
>>>> On Wed, Jan 26, 2011 at 6:46 PM, Tsz Wo (Nicholas), Sze
>>>> <s29752-hadoopu...@yahoo.com> wrote:
>>>>> Okay, I got it now. You were talking about your programs but not the
>>>>> PiEstimator example came from Hadoop. Then, you have to set
>>>>> "mapred.output.key.class" and "mapred.output.value.class" as Srihari
>>>>> mentioned. Below are the APIs.
>>>>>
>>>>> //new API
>>>>> final Job job = ...
>>>>> job.setMapOutputKeyClass(BooleanWritable.class);
>>>>> job.setMapOutputValueClass(LongWritable.class);
>>>>>
>>>>> //old API
>>>>> final JobConf jobconf = ...
>>>>> jobconf.setOutputKeyClass(BooleanWritable.class);
>>>>> jobconf.setOutputValueClass(LongWritable.class);
>>>>>
>>>>> Nicholas
>>>>>
>>>>> ________________________________
>>>>> From: Srihari Anantha Padmanabhan <sriha...@yahoo-inc.com>
>>>>> To: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
>>>>> Sent: Wed, January 26, 2011 10:36:09 AM
>>>>> Subject: Re: PiEstimator error - Type mismatch in key from map
>>>>>
>>>>> I am using Hadoop 0.20.2. I just wrote my own map-reduce program based on
>>>>> the map-reduce tutorial at
>>>>> http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html
>>>>>
>>>>> On Jan 26, 2011, at 10:27 AM, Pedro Costa wrote:
>>>>>
>>>>>> Hadoop 20.1
>>>>>>
>>>>>> On Wed, Jan 26, 2011 at 6:26 PM, Tsz Wo (Nicholas), Sze
>>>>>> <s29752-hadoopu...@yahoo.com> wrote:
>>>>>>> Hi Srihari,
>>>>>>>
>>>>>>> Same questions to you: Which version of Hadoop are you using? And where
>>>>>>> did
>>>>>>> you get the examples? I guess you were able to reproduce it. I suspect
>>>>>>> the
>>>>>>> examples and the Hadoop are in different versions.
>>>>>>>
>>>>>>> Nicholas
>>>>>>>
>>>>>>>
>>>>>>> ________________________________
>>>>>>> From: Srihari Anantha Padmanabhan <sriha...@yahoo-inc.com>
>>>>>>> To: "mapreduce-user@hadoop.apache.org"
>>>>>>> <mapreduce-user@hadoop.apache.org>
>>>>>>> Sent: Wed, January 26, 2011 10:15:08 AM
>>>>>>> Subject: Re: PiEstimator error - Type mismatch in key from map
>>>>>>>
>>>>>>> I got a similar error before in one of my projects. I had to set the
>>>>>>> values
>>>>>>> for "mapred.output.key.class" and "mapred.output.value.class".
>>>>>>> That resolved the issue for me.
>>>>>>> Srihari
>>>>>>> On Jan 26, 2011, at 10:09 AM, Pedro Costa wrote:
>>>>>>>
>>>>>>> Yes, I can reproduce it deterministically. But, I also did some
>>>>>>> changes to the Hadoop MR code. Most definitely this is the reason. I'm
>>>>>>> looking throughly through the code.
>>>>>>>
>>>>>>> I'll say something after I find the problem.
>>>>>>>
>>>>>>> I was just wondering if this error has happened to someone before.
>>>>>>> Maybe I could get a hint and try to see what's my problem easily.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> On Wed, Jan 26, 2011 at 6:02 PM, Tsz Wo (Nicholas), Sze
>>>>>>> <s29752-hadoopu...@yahoo.com> wrote:
>>>>>>>
>>>>>>> Hi Pedro,
>>>>>>>
>>>>>>> This is interesting. Which version of Hadoop are you using? And where
>>>>>>> did
>>>>>>>
>>>>>>> you get the example class files? Also, are you able to reproduce it
>>>>>>>
>>>>>>> deterministically?
>>>>>>>
>>>>>>> Nicholas
>>>>>>>
>>>>>>> ________________________________
>>>>>>>
>>>>>>> From: Pedro Costa <psdc1...@gmail.com>
>>>>>>>
>>>>>>> To: mapreduce-user@hadoop.apache.org
>>>>>>>
>>>>>>> Sent: Wed, January 26, 2011 5:47:01 AM
>>>>>>>
>>>>>>> Subject: PiEstimator error - Type mismatch in key from map
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I run the PI example of hadoop, and I've got the following error:
>>>>>>>
>>>>>>> [code]
>>>>>>>
>>>>>>> java.io.IOException: Type mismatch in key from map: expected
>>>>>>>
>>>>>>> org.apache.hadoop.io.BooleanWritable, recieved
>>>>>>>
>>>>>>> org.apache.hadoop.io.LongWritable
>>>>>>>
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:885)
>>>>>>>
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:551)
>>>>>>>
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:81)
>>>>>>>
>>>>>>> at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
>>>>>>>
>>>>>>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>>>>>>
>>>>>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:637)
>>>>>>>
>>>>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>>>>>>>
>>>>>>> at org.apache.hadoop.mapred.Child.main(Child.java:190)
>>>>>>>
>>>>>>> [/code]
>>>>>>>
>>>>>>> I've look at the map function of the class "PiEstimator.class" and it
>>>>>>> seems
>>>>>>>
>>>>>>> ok.
>>>>>>>
>>>>>>> [code]
>>>>>>>
>>>>>>> public void map(LongWritable offset,
>>>>>>>
>>>>>>> LongWritable size,
>>>>>>>
>>>>>>> OutputCollector<BooleanWritable, LongWritable> out,
>>>>>>>
>>>>>>> Reporter reporter) throws IOException {}
>>>>>>>
>>>>>>> [/code]
>>>>>>>
>>>>>>>
>>>>>>> What's wrong with this examples?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Pedro
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Pedro
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Pedro
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Pedro
>>>>
>>>
>>>
>>>
>>> --
>>> Pedro
>>
>
>
>
> --
> Pedro
>
--
Pedro
