Hi Mark,
Check:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.html
I think that setMaxInputSplitSize(Job job,
long size)
will do what you need.
Regards,
P.A.
> Date: Mon, 6 Jun 2011 19:31:17 -0700
> Subject: Reducing Mapper InputSplit size
> From: [email protected]
> To: [email protected]
>
> Hi,
>
> Does anyone have a way to reduce InputSplit size in general ?
>
> By default, the minimum size chunk that map input should be split into is
> set to 0 (ie.mapred.min.split.size). Can I change dfs.block.size or some
> other configuration to reduce the split size and spawn many mappers?
>
> Thanks,
> Mark
