That doesn't sound right, if you are indeed making use of the new feature. First off, this is not the default behaviour in the actual Apache Hadoop 2.5.0 release. Are you using a different distribution based on that release instead? Secondly, if it were using the auto-calculate feature, it'd only count primary cores as vcores by default.
Are you certain you've looked at the right yarn-site.xml that pertains to the NodeManager, and not a client-purpose file which may simply lack the config mentioned in it? On Sat, Jul 18, 2015 at 7:50 PM Shushant Arora <[email protected]> wrote: > Its hadoop 2.5.0. > Whats the logic of default using hardware detection. Say My node has 8 > actual core and 32 virtual cores. Its taking 26 as value of vcores > available of this node on RM UI. > > On Sat, Jul 18, 2015 at 7:22 PM, Harsh J <[email protected]> wrote: > >> What version of Apache Hadoop are you running? Recent changes have made >> YARN to auto-compute this via hardware detection, by default (rather than >> the 8 default). >> >> On Fri, Jul 17, 2015 at 11:31 PM Shushant Arora < >> [email protected]> wrote: >> >>> In Yarn there is a setting to specify no of vcores that can be allocated >>> to containers. >>> >>> yarn.nodemanager.resource.cpu-vcores >>> >>> In my cluster's nodes yarn-site.xml, this property is not specified. But >>> total vcores displayed on RM's web page for the nodes are different than >>> the default(8). Is there any other place to control this number ? >>> >> >
