Okay, "*" or '*' works fine, which is slightly less horrible, so I can update the myriad-config-default.yml accordingly to put quotes around * for frameworkRole.
--John On Wed, Aug 17, 2016 at 3:44 PM, John Yost <hokiege...@gmail.com> wrote: > The * is causing error within the yaml parsing (see yaml special > characters issues @http://bit.ly/2b507G3) I "fixed" this by setting > frameworkRole = /* and then updating MyriadConfiguration.getFrameworkRole() > to strip off the /. Blech. We can do this or put in a constant like > ANY_ROLE or something like that. Since this is a yaml thing, I guess it's > okay to do /*. Again, just kinda yucky. Darin -> what do you think? > > --John > > On Wed, Aug 17, 2016 at 3:25 PM, Matthew J. Loppatto < > mloppa...@keywcorp.com> wrote: > >> Ah that worked! I'll let you know if I run into any more issues but it >> looks like its good now. Thanks for the help! >> >> Matt >> >> -----Original Message----- >> From: Darin Johnson [mailto:dbjohnson1...@gmail.com] >> Sent: Wednesday, August 17, 2016 3:18 PM >> To: Dev >> Subject: Re: Resource manager error >> >> Take a look at your myriad configuration under yarnEnvironment. You can >> set JAVA_HOME there, should solve the issue. See below. >> yarnEnvironment: >> YARN_HOME: /usr/local/hadoop >> #HADOOP_CONF_DIR=config >> #HADOOP_TMP_DIR=$MESOS_SANDBOX >> #YARN_HOME: hadoop-2.7.0 #this should be relative if nodeManagerUri is set >> #JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes >> necessary >> #JAVA_HOME: jre1.7.0_76 # Path to JRE distribution, relative to sandbox >> directory >> #JAVA_LIBRARY_PATH: /opt/mycompany/lib >> >> On Wed, Aug 17, 2016 at 3:13 PM, Matthew J. Loppatto < >> mloppa...@keywcorp.com >> > wrote: >> >> > I'm running the resource manager as the root user. Checking a few of >> > my nodes, JAVA_HOME is set on all of them for the root env. Am I ok >> > to be using openjdk1.7 or do I have to use Oracle jdk? >> > >> > Matt >> > >> > -----Original Message----- >> > From: John Yost [mailto:hokiege...@gmail.com] >> > Sent: Wednesday, August 17, 2016 3:01 PM >> > To: dev@myriad.incubator.apache.org >> > Subject: Re: Resource manager error >> > >> > Progress is nice! What user are you running myriad as? root? yarn? If >> > it is the former and you are running via sudo, I've seen this type of >> error. >> > If so, sudo to the root user and then launch. Otherwise, please type >> > in env if you are on linux box and confirm you see JAVA_HOME for the >> > user you are launching myriad as. >> > >> > --John >> > >> > On Wed, Aug 17, 2016 at 2:56 PM, Matthew J. Loppatto < >> > mloppa...@keywcorp.com >> > > wrote: >> > >> > > Hey John, >> > > >> > > I set up a role for myriad, restarted mesos-master, and now I'm >> > > seeing RMs starting on the Mesos UI, but they fail with the message >> > > "lost with exit >> > > status: 256". The executor log says "Error: JAVA_HOME is not set >> > > and could not be found." $JAVA_HOME is set on all my slaves as far >> > > as I'm >> > aware. >> > > Running `java -version` confirms openjdk 1.7.0_111. Looks like its >> > > close to a working state. Am I missing something? >> > > >> > > Thanks! >> > > Matt >> > > >> > > -----Original Message----- >> > > From: John Yost [mailto:hokiege...@gmail.com] >> > > Sent: Wednesday, August 17, 2016 2:38 PM >> > > To: dev@myriad.incubator.apache.org >> > > Subject: Re: Resource manager error >> > > >> > > Please uncomment frameworkRole and then add the name of whatever >> > > Mesos role you have configured that is not *. Note: at the risk of >> > > telling you something you already know, you define roles in >> > /etc/mesos-master/roles. >> > > >> > > In the meantime, I opened up a JIRA ticket and gonna fix this ASAP >> > > starting now! :) >> > > >> > > --John >> > > >> > > On Wed, Aug 17, 2016 at 2:23 PM, Matthew J. Loppatto < >> > > mloppa...@keywcorp.com >> > > > wrote: >> > > >> > > > Hey Darin, >> > > > >> > > > Commenting out myriadFrameworkRole got rid of the log message >> > > > about the missing role, but I'm still seeing the "n must be >> positive" >> > > exception. >> > > > >> > > > The only other thing of interest I see in the log is WARN fair. >> > > AllocationFileLoaderService: >> > > > fair-scheduler.xml not found on the classpath. Not sure if that >> > > > is causing any issue though. >> > > > >> > > > Matt >> > > > >> > > > -----Original Message----- >> > > > From: Darin Johnson [mailto:dbjohnson1...@gmail.com] >> > > > Sent: Wednesday, August 17, 2016 1:26 PM >> > > > To: Dev >> > > > Subject: Re: Resource manager error >> > > > >> > > > Hey Matt, >> > > > >> > > > Looking through the code, I think setting myriadFrameworkRole to "*" >> > > > might be the problem. Can you try commenting out that line in >> > > > your config? I'll double check this in a little while too. If >> > > > that works I'll submit a patch that checks that. >> > > > >> > > > Sorry - Myriad is still a pretty young project! Thanks for >> > > > checking it out though! >> > > > >> > > > Darin >> > > > >> > > > On Wed, Aug 17, 2016 at 11:25 AM, Matthew J. Loppatto < >> > > > mloppa...@keywcorp.com> wrote: >> > > > >> > > > > Hey Darin, >> > > > > >> > > > > Pulling from master got rid of the errors I was seeing, however >> > > > > I'm running into a new issue. After starting the resource >> > > > > manager, I see this in the logs: >> > > > > >> > > > > 2016-08-17 10:56:40,709 INFO org.apache.myriad.Main: Launching 1 >> > > > > NM(s) with profile medium >> > > > > 2016-08-17 10:56:40,710 INFO org.apache.myriad.scheduler. >> > > > MyriadOperations: >> > > > > Adding 1 NM instances to cluster >> > > > > 2016-08-17 10:56:40,733 ERROR org.apache.myriad.scheduler. >> > > > event.handlers.ErrorEventHandler: >> > > > > Role '' is not present in the master's --roles >> > > > > >> > > > > My Mesos cluster has the default "*" role so I tried setting >> > > > > frameworkRole: "*" in myriad-config-default.yml, restarted the >> > > > > resource manager and got this error: >> > > > > >> > > > > 2016-08-17 11:06:28,244 ERROR org.apache.myriad.scheduler. >> > > > event.handlers.ResourceOffersEventHandler: >> > > > > Exception thrown while trying to create a task for nm >> > > > > java.lang.IllegalArgumentException: n must be positive >> > > > > at java.util.Random.nextInt(Random.java:300) >> > > > > at org.apache.myriad.scheduler.resource.RangeResource. >> > > > > getRandomValues(RangeResource.java:128) >> > > > > at org.apache.myriad.scheduler.resource.RangeResource. >> > > > > consumeResource(RangeResource.java:99) >> > > > > at org.apache.myriad.scheduler.re >> source.ResourceOfferContainer. >> > > > > consumePorts(ResourceOfferContainer.java:171) >> > > > > at org.apache.myriad.scheduler.NMTaskFactory.createTask( >> > > > > NMTaskFactory.java:45) >> > > > > at org.apache.myriad.scheduler.event.handlers. >> > > > > ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler. >> > > java:119) >> > > > > at org.apache.myriad.scheduler.event.handlers. >> > > > > ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler. >> > java:49) >> > > > > at com.lmax.disruptor.BatchEventProcessor.run( >> > > > > BatchEventProcessor.java:128) >> > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker( >> > > > > ThreadPoolExecutor.java:1145) >> > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( >> > > > > ThreadPoolExecutor.java:615) >> > > > > at java.lang.Thread.run(Thread.java:745) >> > > > > >> > > > > Does Myriad require its own role in Mesos? >> > > > > >> > > > > Thanks, >> > > > > Matt >> > > > > >> > > > > >> > > > > -----Original Message----- >> > > > > From: Darin Johnson [mailto:dbjohnson1...@gmail.com] >> > > > > Sent: Tuesday, August 16, 2016 6:18 PM >> > > > > To: Dev >> > > > > Subject: Re: Resource manager error >> > > > > >> > > > > Hey Mathew, my coworker found the same issue recently, I fixed >> > > > > it on my last pull request, if you'd like to pull from master. >> > > > > >> > > > > Alternatively, you could comment out the appendCgroups line in >> > > > > myriad-scheduler >> > > > > <https://urldefense.proofpoint.com/v2/url?u=https- >> > > > > 3A__github.com_apache_incubator-2Dmyriad_tree_0.2.x_ >> > > > > myriad-2Dscheduler&d=CwIFaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsY >> > > > > HyolgGeY2ZhlU&r=D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s=5zc_OUK_ >> > > > > qwnVQoC2kVCcAgvb4ZmZrVKF-iHca_dif4Y&e= >/src >> > > > > <https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense >> > > > > &d >> > > > > =C >> > > > > wI >> > > > > FaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r=D2bc6ANY3sIF >> > > > > Sx >> > > > > aG >> > > > > Dn >> > > > > Pt52V5PqDlabKIPtzNhAIfJCs&m=sX9u4FJdfE4P4b24cRwGMPuyeT4XkeQRP5t8 >> > > > > wC ZT jV w&s=DeQq-jARIja9dGEYfjeIQMd6jGkf_tNUyvQn7PIMieU&e= .> >> > > > > proofpoint.com/v2/url?u=https-3A__github.com_apache_ >> > > > > incubator-2Dmyriad_tree_0.2.x_myriad-2Dscheduler_src&d=CwIFaQ&c= >> > > > > 31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r= >> > > > > D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s= >> > > > > 6CJWEHP2t7cY2oTmNz9Aq9AV39VEkUOKpMoRuz1q9nY&e= > /main < >> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_ >> > > > > ap >> > > > > ac >> > > > > he >> > > > > _ >> > > > > incubator-2Dmyriad_tree_0.2.x_myriad-2Dscheduler_src_main&d=CwIF >> > > > > aQ &c = 31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r= >> > > > > D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s= >> > > > > BtYphvuvvNZ5owUTfiRd4hW90jq0Ib8GGtKiHU0fTB4&e= > /java < >> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_ >> > > > > ap >> > > > > ac >> > > > > he >> > > > > _ >> > > > > incubator-2Dmyriad_tree_0.2.x_myriad-2Dscheduler_src_main_java&d >> > > > > =C wI Fa Q&c= 31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r= >> > > > > D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s=5o45_ >> > > > > 8zSN96rSaQJ8oCWfhCvmqhSbLpz9fMnV9Fk4WI&e= > /org >> > > > > <https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense >> > > > > &d >> > > > > =C >> > > > > wI >> > > > > FaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r=D2bc6ANY3sIF >> > > > > Sx >> > > > > aG >> > > > > Dn >> > > > > Pt52V5PqDlabKIPtzNhAIfJCs&m=sX9u4FJdfE4P4b24cRwGMPuyeT4XkeQRP5t8 >> > > > > wC ZT jV w&s=DeQq-jARIja9dGEYfjeIQMd6jGkf_tNUyvQn7PIMieU&e= .> >> > > > > proofpoint.com/v2/url?u=https-3A__github.com_apache_ >> > > > > incubator-2Dmyriad_tree_0.2.x_myriad-2Dscheduler_src_main_ >> > > > > java_org&d=CwIFaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU& >> > > > > r= D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s= >> > > > > KUTlDXsl6Okj5nCNNyCqnHcE2ePwEEOsYkf2ASzQP2Y&e= > /apache < >> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_ >> > > > > ap ac he _ >> > > > > incubator-2Dmyriad_tree_0.2.x_myriad-2Dscheduler_src_main_ >> > > > > java_org_apache&d=CwIFaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGe >> > > > > Y2 Zh lU &r= D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s=465af32H1JItcea_ >> > > > > tp5hz7zxwpqWgAqbVA8APaWmSUE&e= > /myriad >> > > > > <https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense >> > > > > &d >> > > > > =C >> > > > > wI >> > > > > FaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r=D2bc6ANY3sIF >> > > > > Sx >> > > > > aG >> > > > > Dn >> > > > > Pt52V5PqDlabKIPtzNhAIfJCs&m=sX9u4FJdfE4P4b24cRwGMPuyeT4XkeQRP5t8 >> > > > > wC ZT jV w&s=DeQq-jARIja9dGEYfjeIQMd6jGkf_tNUyvQn7PIMieU&e= .> >> > > > > proofpoint.com/v2/url?u=https-3A__github.com_apache_ >> > > > > incubator-2Dmyriad_tree_0.2.x_myriad-2Dscheduler_src_main_ >> > > > > java_org_apache_myriad&d=CwIFaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsY >> > > > > HyolgGeY2ZhlU&r=D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s=QEZka3G6qwIsYzvPtX >> > > > > x- w4uVek0Bt2D3bD4M4160Dnk&e= > /scheduler >> > > > > <https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense >> > > > > &d >> > > > > =C >> > > > > wI >> > > > > FaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r=D2bc6ANY3sIF >> > > > > Sx >> > > > > aG >> > > > > Dn >> > > > > Pt52V5PqDlabKIPtzNhAIfJCs&m=sX9u4FJdfE4P4b24cRwGMPuyeT4XkeQRP5t8 >> > > > > wC ZT jV w&s=DeQq-jARIja9dGEYfjeIQMd6jGkf_tNUyvQn7PIMieU&e= .> >> > > > > proofpoint.com/v2/url?u=https-3A__github.com_apache_ >> > > > > incubator-2Dmyriad_tree_0.2.x_myriad-2Dscheduler_src_main_ >> > > > > java_org_apache_myriad_scheduler&d=CwIFaQ&c=31nHN1tvZeuWBT6LwDN4 >> > > > > Ng >> > > > > k1 >> > > > > qe zfsY >> > > > > HyolgGeY2ZhlU&r=D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m= >> > > > > ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s= >> > > > > 2EubhJ2JLwuGjY6DBZXpauvyuXJ0xefgOFHC8lEo5JE&e= > >> > > > > /*NMExecutorCLGenImpl* and rebuild. >> > > > > >> > > > > Sorry that missed my QA unfortunately I'm always using cgroups >> > > > > and didn't test that. We may do a 0.2.1 release but I can say >> when. >> > > > > >> > > > > Darin >> > > > > >> > > > > On Aug 16, 2016 8:49 AM, "Matthew J. Loppatto" >> > > > > <mloppa...@keywcorp.com> >> > > > > wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > >> > > > > > >> > > > > > I’m setting up Myriad 0.2.0 on my Mesos cluster following this >> > guide: >> > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a >> pache. >> > > > > > or >> > > > > > g_ >> > > > > > confluence_display_MYRIAD_&d=CwIFaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1 >> > > > > > qe >> > > > > > zf >> > > > > > sY >> > > > > > Hy >> > > > > > olgGeY2ZhlU&r=D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m=ib >> > > > > > xh >> > > > > > OZ >> > > > > > QS >> > > > > > sK >> > > > > > tyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s=LaQad9p3ZI3Rt5cTn3kHAb58Bu >> > > > > > SD >> > > > > > 5e >> > > > > > tw >> > > > > > Im >> > > > > > WZHzFz6Sk&e= >> > > > > > Installing+for+Developers >> > > > > > >> > > > > > >> > > > > > >> > > > > > And I get the following error in the resource manager executor >> > > > > > log in mesos after starting it with >> > > > > > `/opt/hadoop-2.7.2/bin/yarn >> > > > > resourcemanager`: >> > > > > > >> > > > > > >> > > > > > >> > > > > > chown: cannot access >> > > > > > ‘/sys/fs/cgroup/cpu/mesos/f5d6c530-c13d-4b1d-bc30- >> > > > > f298affb6442’: >> > > > > > No such file or directory >> > > > > > >> > > > > > env: /bin/yarn: No such file or directory >> > > > > > >> > > > > > ory >> > > > > > >> > > > > > >> > > > > > >> > > > > > It appears the ‘mesos’ directory doesn’t exist under >> > > > /sys/fs/cgroup/cpu. >> > > > > > Any ideas what the issue could be? >> > > > > > >> > > > > > >> > > > > > >> > > > > > This is my yarn-site.xml: >> > > > > > >> > > > > > >> > > > > > >> > > > > > <configuration> >> > > > > > >> > > > > > <!-- Site-specific YARN configuration properties --> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.aux-services</name> >> > > > > > >> > > > > > <value>mapreduce_shuffle,myriad_executor</value> >> > > > > > >> > > > > > <!-- If using MapR distro, please use the following >> value: >> > > > > > >> > > > > > >> > > > > > <value>mapreduce_shuffle,mapr_direct_shuffle,myriad_executor</ >> > > > > > va >> > > > > > lu >> > > > > > e> >> > > > > > --> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > >> > > > > > <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</n >> > > > > > am >> > > > > > e> >> > > > > > >> > > > > > <value>org.apache.hadoop.mapred.ShuffleHandler</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > >> > > > > > <name>yarn.nodemanager.aux-services.myriad_executor.class</nam >> > > > > > e> >> > > > > > >> > > > > > >> > > > > > <value>org.apache.myriad.executor.MyriadExecutorAuxService</va >> > > > > > lu >> > > > > > e> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > >> > > > > > <name>yarn.nm.liveness-monitor.expiry-interval-ms</name> >> > > > > > >> > > > > > <value>2000</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > >> > > > > > <name>yarn.am.liveness-monitor.expiry-interval-ms</name> >> > > > > > >> > > > > > <value>10000</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > >> > > > > > <name>yarn.resourcemanager.nm.liveness-monitor.interval-ms</na >> > > > > > me >> > > > > > > >> > > > > > >> > > > > > <value>1000</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <!-- Needed for Fine Grain Scaling --> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.scheduler.minimum-allocation-vcores</name> >> > > > > > >> > > > > > <value>0</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.scheduler.minimum-allocation-mb</name> >> > > > > > >> > > > > > <value>0</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <!-- Site specific YARN configuration properties --> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.resource.cpu-vcores</name> >> > > > > > >> > > > > > <value>${nodemanager.resource.cpu-vcores}</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.resource.memory-mb</name> >> > > > > > >> > > > > > <value>${nodemanager.resource.memory-mb}</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <!--These options enable dynamic port assignment by mesos --> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.address</name> >> > > > > > >> > > > > > <value>${myriad.yarn.nodemanager.address}</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.webapp.address</name> >> > > > > > >> > > > > > <value>${myriad.yarn.nodemanager.webapp.address}</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.webapp.https.address</name> >> > > > > > >> > > > > > <value>${myriad.yarn.nodemanager.webapp.address}</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.localizer.address</name> >> > > > > > >> > > > > > <value>${myriad.yarn.nodemanager.localizer.address}</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <!-- Configure Myriad Scheduler here --> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.resourcemanager.scheduler.class</name> >> > > > > > >> > > > > > >> > > > > > <value>org.apache.myriad.scheduler.yarn.MyriadFairScheduler</v >> > > > > > al >> > > > > > ue >> > > > > > > >> > > > > > >> > > > > > <description>One can configure other scehdulers as well >> > > > > > from following >> > > > > > list: >> > > > > > org.apache.myriad.scheduler.yarn.MyriadCapacityScheduler, >> > > > > > org.apache.myriad.scheduler.yarn.MyriadFifoScheduler</descript >> > > > > > io >> > > > > > n> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <!-- Disable PMem/VMem checks for Hadoop 2.7.2 --> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.pmem-check-enabled</name> >> > > > > > >> > > > > > <value>false</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > <property> >> > > > > > >> > > > > > <name>yarn.nodemanager.vmem-check-enabled</name> >> > > > > > >> > > > > > <value>false</value> >> > > > > > >> > > > > > </property> >> > > > > > >> > > > > > </configuration> >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > My myriad-config-default.yml: >> > > > > > >> > > > > > >> > > > > > >> > > > > > mesosMaster: zk://myip:2181/mesos >> > > > > > >> > > > > > checkpoint: false >> > > > > > >> > > > > > frameworkFailoverTimeout: 43200000 >> > > > > > >> > > > > > frameworkName: MyriadAlpha >> > > > > > >> > > > > > frameworkRole: >> > > > > > >> > > > > > frameworkUser: root # User the Node Manager runs as, >> required >> > if >> > > > > > nodeManagerURI set, otherwise defaults to the user >> > > > > > >> > > > > > # running the resource manager. >> > > > > > >> > > > > > frameworkSuperUser: root # To be depricated, currently >> > > > > > permissions need set by a superuser due to Mesos-1790. Must >> > > > > > be >> > > > > > >> > > > > > # root or have passwordless sudo. >> > > > > > Required if nodeManagerURI set, ignored otherwise. >> > > > > > >> > > > > > nativeLibrary: /usr/local/lib/libmesos.so >> > > > > > >> > > > > > zkServers: myip:2181 >> > > > > > >> > > > > > zkTimeout: 20000 >> > > > > > >> > > > > > restApiPort: 8192 >> > > > > > >> > > > > > servedConfigPath: dist/config.tgz >> > > > > > >> > > > > > servedBinaryPath: dist/binary.tgz >> > > > > > >> > > > > > profiles: >> > > > > > >> > > > > > zero: # NMs launched with this profile dynamically obtain >> > > > > > cpu/mem from Mesos >> > > > > > >> > > > > > cpu: 0 >> > > > > > >> > > > > > mem: 0 >> > > > > > >> > > > > > small: >> > > > > > >> > > > > > cpu: 2 >> > > > > > >> > > > > > mem: 2048 >> > > > > > >> > > > > > medium: >> > > > > > >> > > > > > cpu: 4 >> > > > > > >> > > > > > mem: 4096 >> > > > > > >> > > > > > large: >> > > > > > >> > > > > > cpu: 10 >> > > > > > >> > > > > > mem: 12288 >> > > > > > >> > > > > > nmInstances: # NMs to start with. Requires at least 1 NM with >> > > > > > a non-zero profile. >> > > > > > >> > > > > > medium: 1 # <profile_name : instances> >> > > > > > >> > > > > > rebalancer: false >> > > > > > >> > > > > > haEnabled: false >> > > > > > >> > > > > > nodemanager: >> > > > > > >> > > > > > jvmMaxMemoryMB: 1024 >> > > > > > >> > > > > > cpus: 0.2 >> > > > > > >> > > > > > cgroups: false >> > > > > > >> > > > > > executor: >> > > > > > >> > > > > > jvmMaxMemoryMB: 256 >> > > > > > >> > > > > > path: >> > > > > > file:///usr/local/libexec/mesos/myriad-executor-runnable-0.1.0 >> > > > > > .j >> > > > > > ar >> > > > > > >> > > > > > #The following should be used for a remotely distributed URI, >> > > > > > hdfs assumed but other URI types valid. >> > > > > > >> > > > > > #nodeManagerUri: hdfs://namenode:port/dist/hadoop-2.7.0.tar.gz >> > > > > > >> > > > > > #configUri: >> > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1_ >> > > > > > ap >> > > > > > i_ >> > > > > > ar >> > > > > > if >> > > > > > acts_config.tgz&d=CwIFaQ&c=31nHN1tvZeuWBT6LwDN4Ngk1qezfsYHyolg >> > > > > > Ge >> > > > > > Y2 >> > > > > > Zh >> > > > > > lU >> > > > > > &r=D2bc6ANY3sIFSxaGDnPt52V5PqDlabKIPtzNhAIfJCs&m=ibxhOZQSsKtyV >> > > > > > i5 >> > > > > > ir >> > > > > > uY >> > > > > > 8I >> > > > > > mkW7bQ8zOrHcuDTLL7GBwA&s=IpOqhUOtwJsdorbAOeoY7GgHalMJ1s9EUjuRU >> > > > > > fR >> > > > > > sm >> > > > > > ew >> > > > > > &e >> > > > > > = >> > > > > > >> > > > > > #jvmUri: >> > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__downloads >> > > > > > .m >> > > > > > yc >> > > > > > om >> > > > > > pa >> > > > > > ny.com_java_jre-2D7u76-2Dlinux-2Dx64.tar.gz&d=CwIFaQ&c=31nHN1t >> > > > > > vZ >> > > > > > eu >> > > > > > WB >> > > > > > T6 >> > > > > > LwDN4Ngk1qezfsYHyolgGeY2ZhlU&r=D2bc6ANY3sIFSxaGDnPt52V5PqDlabK >> > > > > > IP >> > > > > > tz >> > > > > > Nh >> > > > > > AI >> > > > > > fJCs&m=ibxhOZQSsKtyVi5iruY8ImkW7bQ8zOrHcuDTLL7GBwA&s=jPB2677RH >> > > > > > 3k >> > > > > > 3C >> > > > > > Ls >> > > > > > gl >> > > > > > 4Zj3tGawuCLVB1a2WXBUOWEelU&e= >> > > > > > >> > > > > > yarnEnvironment: >> > > > > > >> > > > > > YARN_HOME: /opt/hadoop-2.7.2 >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > Thanks! >> > > > > > >> > > > > > Matt >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
