Tao Jie created MYRIAD-251:
------------------------------
Summary: ZERO size NodeManager fail to obtain resource from Mesos
Offer
Key: MYRIAD-251
URL: https://issues.apache.org/jira/browse/MYRIAD-251
Project: Myriad
Issue Type: Bug
Components: Scheduler
Reporter: Tao Jie
I tried Fine-grained Scaling and flexed up zero size NodeManager, then I run a
MR job which request for resource.
However zero size NM did not obtain resource from mesos offer. RM logs like:
{code}
2016-12-14 16:58:23,929 INFO
org.apache.myriad.scheduler.fgs.NMHeartBeatHandler: Did not update
bdi13.cmss.com with 10 cores and 5888 memory, over max cpu cores and/or max
memory
2016-12-14 16:58:23,931 WARN
org.apache.myriad.scheduler.fgs.YarnNodeCapacityManager: Asked to set Node
bdi13.cmss.com:31905 to a value less than zero! Had <memory:0, vCores:0>,
setting to <memory:0, vCores:0>.
2016-12-14 16:58:23,931 WARN
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler:
Update resource on node: bdi13.cmss.com with the same resource: <memory:0,
vCores:0>
{code}
It seems that mesos offer with memory larger than 2252.8mb would be denied, and
2252.8mb is fixed value in code :
{code}
private Double generateNodeManagerMemory() {
return (NodeManagerConfiguration.DEFAULT_JVM_MAX_MEMORY_MB) * (1 +
NodeManagerConfiguration.JVM_OVERHEAD);
}
{code}
where DEFAULT_JVM_MAX_MEMORY_MB=2048 and
NodeManagerConfiguration.JVM_OVERHEAD=0.1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
