OK edited, Ruth I decided to do the absolute path of /opt/hadoop-2.7.1 vs $YARN_HOME, in case the environment variable was off and someone copy/pasted.
Still need to add the instructions for running the RM in Marathon. Darin On Mon, Oct 5, 2015 at 4:39 PM, Darin Johnson <[email protected]> wrote: > sudo rm $YARN_HOME/etc/hadoop/yarn-site.xml <- This is slightly off, I'll > go correct. > > On Mon, Oct 5, 2015 at 4:38 PM, Darin Johnson <[email protected]> > wrote: > >> John, I'm running off: https://github.com/apache/incubator-myriad, it >> seems to run OK there's a couple of NPE issues though they're rare events >> (I had to invent a way to make one occur), I've got a PR for one if you >> want to test/review it for me :). >> >> I'll try to update the wiki with instructions on running the resource >> manager via marathon. >> >> Darin >> >> >> On Mon, Oct 5, 2015 at 4:27 PM, John Omernik <[email protected]> wrote: >> >>> I see. That makes sense. Thanks for the tip. >>> >>> Is it safe to pull down a recent version at this point? Are we using the >>> official "master" or phase1? (the lazy man in me is asking for a link to >>> the current repo so I don't have to read back over emails to see where I >>> should go :) >>> >>> >>> >>> On Mon, Oct 5, 2015 at 3:25 PM, Darin Johnson <[email protected]> >>> wrote: >>> >>> > Hey John, >>> > >>> > Are you trying to run the resource manager from the tar ball via >>> marathon? >>> > It's doable, my suggested approach would be to use a json like this: >>> > >>> > { >>> > "id": "resource-manager", >>> > "uris": ["hdfs://namenode:port/dist/hadoop-2.7.0.tgz", >>> > "hdfs://namenode:port/dist/conf/hadoop/yarn-site.xml", >>> > "hdfs://namenode:port/dist/conf/hadoop/hdfs-site.xml", >>> > "hdfs:///dist/conf/hadoop/core-site.xml", >>> > "hdfs://namenode:port/dist/conf/hadoop/mapred-site.xml"], >>> > "cmd": "cp *.xml hadoop-2.7.0/etc/hadoop && cd hadoop-2.7.0 && >>> bin/yarn >>> > resourcemanager", >>> > "mem": 16, >>> > "cpu": 1 >>> > "instances" : 1, >>> > "user": "yarn" >>> > } >>> > >>> > Basically it keeps you from redoing the tar ball every time you edit a >>> > config, instead you just upload the new yarn-site.xml. The Node >>> Manager >>> > gets it's config from the Resource Manager (I'm assuming this is all >>> for >>> > remote distribution, otherwise creating the tar ball is optional). >>> > >>> > Darin >>> > >>> > On Mon, Oct 5, 2015 at 2:36 PM, John Omernik <[email protected]> wrote: >>> > >>> > > Hey all, I've been waiting until the chaos of the code move has died >>> > down. >>> > > I am looking to get this working on my MapR cluster now, and would >>> like >>> > > some clarification on instructions here: >>> > > >>> > > >>> > > >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/MYRIAD/Installing+for+Administrators >>> > > >>> > > Basically, in the instructions below, it has the "remove the >>> > > yarn-site.xml. Yet to run the resource manager with myriad, you >>> need the >>> > > yarn-site to be packaged with things (unless I am reading that >>> > incorrectly) >>> > > Is the only option right now to created a tarball for nodemanagers, >>> and >>> > > have this be different from the tarball for the resource manager? >>> > > >>> > > Step 5: Create the Tarball >>> > > >>> > > The tarball has all of the files needed for the Node Managers and >>> > Resource >>> > > Managers. The following shows how to create the tarball and place it >>> in >>> > > HDFS: >>> > > cd ~ >>> > > sudo cp -rp $YARN_HOME . >>> > > sudo rm $YARN_HOME/etc/hadoop/yarn-site.xml >>> > > sudo tar -zcpf ~/hadoop-2.7.1.tar.gz hadoop-2.7.1 >>> > > hadoop fs -put ~/hadoop-2.7.1.tar.gz /dist >>> > > >>> > >>> >> >> >
