Varun, Thanks for the reply. I undestand the arn.scheduler.maximum- allocation-vcores parameter. I just asking why the default parameter is yarn.scheduler.maximum-allocation-vcores=32. And yarn.nodemanager.resource.cpu-vcores=8.
In my opinion, if the yarn.scheduler.maximun-allocation-vcore is 32 tby default the yarn.nodemanager.resource.cpu-vcores would be equal or greater than 32, by default. Is this make sense? On Sun, Aug 23, 2015 at 2:00 PM, Varun Saxena <[email protected]> wrote: > Hi Pedro, > > Actual allocation would depend on the total resource capability advertised > by NM while registering with RM. > > yarn.scheduler.maximum-allocation-vcores merely puts an upper cap on number > of vcores which can be allocated by RM i.e. any Resource request/ask from AM > which asks for vcores > 32(default value) for a container, will be normalized > back to 32. > > If there is no such node available, this allocation will not be fulfilled. > > yarn.scheduler.maximum-allocation-vcores will be configured in resource > manager and hence will be common for a cluster which can possibly have > multiple nodes with heterogeneous resource capabilities > > yarn.nodemanager.resource.cpu-vcores on the other hand will have to be > configured as per resource capability of that particular node. > > Recently there has been work done to automatically get memory and CPU > information from underlying OS(supported OS being Linux and Windows) if > configured to do so. This change would be available in 2.8 > I hope this answers your question. > > Regards, > Varun Saxena. > > On Sun, Aug 23, 2015 at 9:40 PM, Pedro Magalhaes <[email protected]> > wrote: > >> I was looking at default parameters for: >> >> yarn.nodemanager.resource.cpu-vcores = 8 >> yarn.scheduler.maximum-allocation-vcores = 32 >> >> For me this two parameters as default doesnt make any sense. >> >> The first one say "the number of CPU cores that can be allocated for >> containers." (I imagine that is vcore) The seconds says: "The maximum >> allocation for every container request at the RM". In my opinion, the >> second one must be equal or less than the first one. >> >> How can allocate 32 vcores for a container if i have only 8 cores >> available per container? >> > >
