[ https://issues.apache.org/jira/browse/MYRIAD-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Santosh Marella closed MYRIAD-125. ---------------------------------- Resolution: Cannot Reproduce > Myriad /config API issue for "YARN_NODEMANAGER_OPTS" param when RM's hostname > is passed. > ---------------------------------------------------------------------------------------- > > Key: MYRIAD-125 > URL: https://issues.apache.org/jira/browse/MYRIAD-125 > Project: Myriad > Issue Type: Bug > Components: Scheduler > Reporter: Sarjeet Singh > Assignee: Santosh Marella > Fix For: Myriad 0.1.0 > > > The issue is with "YARN_NODEMANAGER_OPTS" param that has > "-Dyarn.resourcemanager.hostname=<host-name>" keep appending on each offer > created by myriad to yarnEnvironment. > Found this piece of code that has the issue. > In TaskFactory.java: > <code> > String rmHostName = System.getProperty(YARN_RESOURCEMANAGER_HOSTNAME); > if (rmHostName != null && !rmHostName.isEmpty()) { > String nmOpts = > nmTaskConfig.getYarnEnvironment().get(YARN_NODEMANAGER_OPTS_KEY); > if (nmOpts == null) { > nmOpts = ""; > } > nmOpts += " " + "-D" + YARN_RESOURCEMANAGER_HOSTNAME + "=" + > rmHostName; > > nmTaskConfig.getYarnEnvironment().put(YARN_NODEMANAGER_OPTS_KEY, nmOpts); > LOGGER.info(YARN_RESOURCEMANAGER_HOSTNAME + " is set to " + > rmHostName + > " via YARN_RESOURCEMANAGER_OPTS. Passing it into > YARN_NODEMANAGER_OPTS."); > } > </code> > Observed this when I tried to start RM from marathon, and HTTP GET the > /config API and checked "YARN_NODEMANAGER_OPTS" between 10-15 min interval . > > Here is what I observed: > "yarnEnvironment": { > "YARN_HOME": "/opt/mapr/hadoop/hadoop-2.7.0/", > "YARN_NODEMANAGER_OPTS": "-Dnodemanager.resource.io-spindles=4.0 > -Dyarn.resourcemanager.hostname=rm.marathon.mesos" > }, > After 10-15 min of being idle: > "yarnEnvironment": { > "YARN_HOME": "/opt/mapr/hadoop/hadoop-2.7.0/", > "YARN_NODEMANAGER_OPTS": "-Dnodemanager.resource.io-spindles=4.0 > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos > -Dyarn.resourcemanager.hostname=rm.marathon.mesos" > }, > Let me know if need any addiotinal detail about the issue? Thanks. > FYI, this should be easy to reproducible by just starting Myriad and check > /config API output between 10-15 min interval. -- This message was sent by Atlassian JIRA (v6.3.4#6332)