Hi Pedro, Real default value of yarn.scheduler.maximum-allocation-vcores is 4. The value of 32 is actually a documentation issue and has been fixed recently.
Regards, Varun Saxena. On Sun, Aug 23, 2015 at 10:39 PM, Pedro Magalhaes <[email protected]> wrote: > Varun, > Thanks for the reply. I undestand the arn.scheduler.maximum- > allocation-vcores parameter. I just asking why the default parameter is > yarn.scheduler.maximum-allocation-vcores=32. And > yarn.nodemanager.resource.cpu-vcores=8. > > In my opinion, if the yarn.scheduler.maximun-allocation-vcore is 32 tby > default the yarn.nodemanager.resource.cpu-vcores would be equal or greater > than 32, by default. > Is this make sense? > > > > > On Sun, Aug 23, 2015 at 2:00 PM, Varun Saxena <[email protected]> > wrote: > >> Hi Pedro, >> >> Actual allocation would depend on the total resource capability >> advertised by NM while registering with RM. >> >> yarn.scheduler.maximum-allocation-vcores merely puts an upper cap on number >> of vcores which can be allocated by RM i.e. any Resource request/ask from AM >> which asks for vcores > 32(default value) for a container, will be >> normalized back to 32. >> >> If there is no such node available, this allocation will not be fulfilled. >> >> yarn.scheduler.maximum-allocation-vcores will be configured in resource >> manager and hence will be common for a cluster which can possibly have >> multiple nodes with heterogeneous resource capabilities >> >> yarn.nodemanager.resource.cpu-vcores on the other hand will have to be >> configured as per resource capability of that particular node. >> >> Recently there has been work done to automatically get memory and CPU >> information from underlying OS(supported OS being Linux and Windows) if >> configured to do so. This change would be available in 2.8 >> I hope this answers your question. >> >> Regards, >> Varun Saxena. >> >> On Sun, Aug 23, 2015 at 9:40 PM, Pedro Magalhaes <[email protected]> >> wrote: >> >>> I was looking at default parameters for: >>> >>> yarn.nodemanager.resource.cpu-vcores = 8 >>> yarn.scheduler.maximum-allocation-vcores = 32 >>> >>> For me this two parameters as default doesnt make any sense. >>> >>> The first one say "the number of CPU cores that can be allocated for >>> containers." (I imagine that is vcore) The seconds says: "The maximum >>> allocation for every container request at the RM". In my opinion, the >>> second one must be equal or less than the first one. >>> >>> How can allocate 32 vcores for a container if i have only 8 cores >>> available per container? >>> >> >> >
