found it
SET mapred.min.split.size 12500000; SET mapred.max.split.size 12500000; SET pig.noSplitCombination true; On Wed, Jan 25, 2012 at 6:06 PM, Yang <[email protected]> wrote: > this question was asked a few days before, after that, I dug around the > source code and found some knobs to turn, > > but after adding the following, I still could not get pig to use smaller > split sizes > > any ideas? > > -- this does not work for pig > -- SET mapred.map.tasks 100; > SET pig.mapsplits.count 100; > SET mapred.max.jobs.per.node 1; > --- following changed after 0.21 > --- SET mapred.min.split.size 100; > SET mapreduce.input.fileinputformat.split.minsize 100; > SET mapreduce.input.fileinputformat.split.maxsize 100; > SET pig.splitCombination false; > SET pig.noSplitCombination true; > SET pig.maxCombinedSplitSize 100; > >
