Hi Kevin, By default, the NodeManager monitors physical and virtual memory usage of containers. Containers that exceed either limit are killed. Admins can disable the checks by setting yarn.nodemanager.pmem-check-enabled and/or yarn.nodemanager.vmem-check-enabled to false. The virtual memory limit for a container is determined using the config variable yarn.nodemanager.vmem-pmem-ratio(default value is 2.1).
In case of vcores - 1. If you’re using Cgroups under LinuxContainerExecutor, by default, if there is spare CPU available on the node, your container will be allowed to use it. Admins can restrict containers to use only the CPU allocated to them by setting yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage to true. This setting is only applicable when using Cgroups under LinuxContainerExecutor. 2. If you aren’t using Cgroups under LinuxContainerExecutor, there is no limiting of the amount of the CPU that containers can use. -Varun From: Kevin Reply-To: "[email protected]<mailto:[email protected]>" Date: Friday, May 22, 2015 at 3:30 AM To: "[email protected]<mailto:[email protected]>" Subject: Using YARN with native applications Hello, I have been using the distributed shell application and Oozie to run native C++ applications in the cluster. Is YARN able to see the resources these native applications use. For example, if I use Oozie's shell action, the NodeManager hosts the mapper container and allocates a certain amount of memory and vcores (as configured). What happens if my C++ application uses more memory or vcores than the NodeManager allocated? I was looking in the Hadoop code and I couldn't find my way to answer. Although, it seems the LinuxContainerExecutor may be the answer to my question since it uses cgroups. I'm interested to know how YARN reacts to non-Java applications running inside of it. Thanks, Kevin
