Hello, In order to track demand for our cluster's resources, I'm trying to figure out a way to measure instantaneous application demand for vcore and memory. The motivation is to know when and by how much we are under- or over-utilizing our resources.
I've tried summing //apps/app/allocatedMB and //apps/app/allocatedVCores from this RM API endpoint: http://<url>/ws/v1/cluster/apps?states=RUNNING This is quite close to what I want, but does not capture the case when total requested resources exceeds what's available -- I think the above sum represents resources allocated to currently running containers and excludes requested resources that are currently not running. I poked around a bit in the source and if I'm interpreting it correctly, it seems like this class has the information I need: https://hadoop.apache.org/docs/r2.5.2/api/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.html#getNeededResources() but I couldn't find where this information was exposed in the YARN rest api/cli (seemed plausible this applied to containers and not applications, but I wasn't sure) Does anyone know how I might accomplish this? Obviously I'm open to alternative approaches as well. Thanks in advance! Chris Mullins
