Great. Just thought I'd be missing something :)
On Mon, Jun 7, 2010 at 01:55, Aaron Kimball <aa...@cloudera.com> wrote:
> Yes. LJR sets your number of reduce tasks to 1 if the number is >= 1.
> Subnote: I've posted a patch to fix this at MAPREDUCE-434, but it's not
> committed.
> - Aaron
>
> On Mon, Jun 7, 2010 at 1:42 AM, Torsten Curdt <tcu...@vafer.org> wrote:
>>
>> I see only one.
>>
>> Could it be that using the LocalJobRunner interferes here?
>>
>> On Mon, Jun 7, 2010 at 01:31, Eric Sammer <esam...@cloudera.com> wrote:
>> > Torsten:
>> >
>> > To clarify, how many reducers do you actually see? (i.e. Do you see 4
>> > reducers or 1?) It should work as you expect.
>> >
>> > On Sun, Jun 6, 2010 at 1:33 PM, Torsten Curdt <tcu...@vafer.org> wrote:
>> >> When I set
>> >>
>> >> job.setPartitionerClass(MyPartitioner.class);
>> >> job.setNumReduceTasks(4);
>> >>
>> >> I would expect to see my MyParitioner get called with
>> >>
>> >> getPartition(key, value, 4)
>> >>
>> >> but still I see it only get called with 1.
>> >>
>> >> If also tried setting
>> >>
>> >> conf.set("mapred.map.tasks.speculative.execution",
>> >> "false");
>> >> conf.set("mapred.reduce.tasks.speculative.execution",
>> >> "false");
>> >>
>> >> What am I missing here?
>> >>
>> >> cheers
>> >> --
>> >> Torsten
>> >>
>> >
>> >
>> >
>> > --
>> > Eric Sammer
>> > phone: +1-917-287-2675
>> > twitter: esammer
>> > data: www.cloudera.com
>> >
>
>
