Well, when I said I found a solution this link was one of them :). Even though I set :
dfs.block.size = mapred.min.split.size = mapred.max.split.size = 14MB the job is still running maps with 64MB ! I don't see what else can I change :( Thanks, Mark On Fri, Oct 26, 2012 at 2:23 PM, Bertrand Dechoux <decho...@gmail.com>wrote: > Hi Mark, > > I think http://wiki.apache.org/hadoop/HowManyMapsAndReduces might interest > you. > If you require more information, feel free to ask after reading it. > > Regards > > Bertrand > > On Fri, Oct 26, 2012 at 10:47 PM, Mark Olimpiati <markq2...@gmail.com > >wrote: > > > Hi, > > > > I've found that the solution to control the split size per mapper is to > > modify the following configurations: > > > > mapred.min.split.size and mapred.max.split.size, but when I set them both > > to 14MB with dfs.block.size = 64MB, the splits are still = 64MB. > > > > So, is there a relation between them that I should consider? > > > > Thank you, > > Mark > > > > > > -- > Bertrand Dechoux >
