I believe they are normalized to be multiples of yarn.scheduler.increment-allocation-mb. yarn.scheduler.minimum-allocation-mb can be set to as low as zero. Llama does this.
As to why normalization, I think it is to make sure there is no external fragmentation. It is similar to why memory is paged. On Wed, Jun 11, 2014 at 5:52 PM, Ashwin Shankar <[email protected]> wrote: > Hi, > Anyone knows why resource requests from AMs are normalized to > be multiples of yarn.scheduler.minimum-allocation-mb which is 1G > by default ? > Also is there any problem with reducing yarn.scheduler.minimum-allocation-mb > to less > than 1G ? > > /** > > * Utility method to normalize a list of resource requests, by insuring > that > > * the memory for each request is a multiple of minMemory and is not > zero. > > */ > > SchedulerUtils.normalizeRequests() > -- > Thanks, > Ashwin > > >
