Re: Number of map task

2014-04-22 Thread Pradeep Gollakota 
Pig is a little too smart when dealing with data. It has a feature called
split combination. If you set it to false, you should see more mappers.

SET pig.noSplitCombination true;



On Tue, Apr 22, 2014 at 12:14 PM, Patcharee Thongtra <
patcharee.thong...@uni.no> wrote:

> Hi,
>
> I wrote a custom InputFormat. When I ran the pig script Load function
> using this InputFormat, the number of InputSplit = 16, but there was only 2
> map tasks handling these splits. Apparently the no. of map tasks = the no.
> of input files.
>
> Does the number of Map task not correspond to the number of splits?
>
> I think the job will be done quicker if there are more Map tasks?
>
> Patcharee
>

Number of map task

2014-04-22 Thread Patcharee Thongtra 

Hi,

I wrote a custom InputFormat. When I ran the pig script Load function 
using this InputFormat, the number of InputSplit = 16, but there was 
only 2 map tasks handling these splits. Apparently the no. of map tasks 
= the no. of input files.


Does the number of Map task not correspond to the number of splits?

I think the job will be done quicker if there are more Map tasks?

Patcharee