Also a side note: The Flexing up and now having to have at least one node manager specified at startup:
nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero profile. medium: 1 # <profile_name : instances> Is going to lead to task failures with mesos dns because the name won't be ready right away (1 minute delay after kicking off Myriad) do we NEED to have a non-0 profile nodemanager startup with the resource manager? On Tue, Sep 8, 2015 at 3:16 PM, John Omernik <j...@omernik.com> wrote: > Cool. Question about the yarn-site.xml in general. > > I was struggling with some things in the wiki on this page: > https://cwiki.apache.org/confluence/display/MYRIAD/Installing+for+Administrators > > Basically in step 5: > Step 5: Configure YARN to use Myriad > > Modify the */opt/hadoop-2.7.0/etc/hadoop/yarn-site.xml* file as > instructed in Sample: myriad-config-default.yml > <https://cwiki.apache.org/confluence/display/MYRIAD/Sample%3A+myriad-config-default.yml> > . > > > (It should not link to the yml, but to the yarn site, side issue) it has > us put that information in the yarn-site.xml This makes sense. The > resource manager needs to be aware of the myriad stuff. > > Then I go to create a tarbal, (which I SHOULD be able to use for both > resource manager and nodemanager... right?) However, the instructions state > to remove the *.xml files. > > Step 6: Create the Tarball > > The tarball has all of the files needed for the Node Managers and > Resource Managers. The following shows how to create the tarball and place > it in HDFS: > cd ~ > sudo cp -rp /opt/hadoop-2.7.0 . > sudo rm hadoop-2.7.0/etc/hadoop/*.xml > sudo tar -zcpf ~/hadoop-2.7.0.tar.gz hadoop-2.7.0 > hadoop fs -put ~/hadoop-2.7.0.tar.gz /dist > > > What I ended up doing... since I am running the resourcemanager (myriad) > in marathon, is I created two tarballs. One is my hadoop-2.7.0-RM.tar.gz > which has the all the xml files still in the tar ball for shipping to > marathon. Then other is hadoop-2.7.0-NM.tar.gz which per the instructions > removes the *.xml files from the /etc/hadoop/ directory. > > > I guess... my logic is that myriad creates the conf directory for the > nodemanagers... but then I thought, and I overthinking something? Am I > missing something? Could that be factoring into what I am doing here? > > > Obviously my first steps are to add the extra yarn-site.xml entries, but > in this current setup, they are only going into the resource manager > yarn-site as the the node-managers don't have a yarn-site in their > directories. > > > > > > > > On Tue, Sep 8, 2015 at 3:09 PM, yuliya Feldman < > yufeld...@yahoo.com.invalid> wrote: > >> Take a look at : https://github.com/mesos/myriad/pull/128 >> for yarn-site.xml updates >> >> From: John Omernik <j...@omernik.com> >> To: dev@myriad.incubator.apache.org >> Sent: Tuesday, September 8, 2015 12:38 PM >> Subject: Getting Nodes to be "Running" in Mesos >> >> So I am playing around with a recent build of Myriad, and I am using MapR >> 5.0 (hadoop-2.7.0) I hate to use the dev list as a "help Myriad won't run" >> forum, so please forgive me if I am using the list wrong. >> >> Basically, I seem to be able to get myriad running, and the things up, and >> it tries to start a nodemanager. >> >> In mesos, the status of the nodemanager task never gets past staging, and >> eventually, fails. The logs for both the node manager and myriad, seem to >> look healthy, and I am not sure where I should look next to troubleshoot >> what is happening. Basically you can see the registration of the >> nodemanager, and then it fails with no error in the logs... Any thoughts >> would be appreciated on where I can look next for troubleshooting. >> >> >> Node Manager Logs (complete) >> >> STARTUP_MSG: build = g...@github.com:mapr/private-hadoop-common.git >> -r fc95119f587541fb3a9af0dbeeed23c974178115; compiled by 'root' on >> 2015-08-19T20:02Z >> STARTUP_MSG: java = 1.8.0_45-internal >> ************************************************************/ >> 15/09/08 14:35:23 INFO nodemanager.NodeManager: registered UNIX signal >> handlers for [TERM, HUP, INT] >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for >> class org.apache.hadoop.yarn.server.nodemanager.NodeManager >> 15/09/08 14:35:24 INFO impl.MetricsConfig: loaded properties from >> hadoop-metrics2.properties >> 15/09/08 14:35:24 INFO impl.MetricsSystemImpl: Scheduled snapshot >> period at 10 second(s). >> 15/09/08 14:35:24 INFO impl.MetricsSystemImpl: NodeManager metrics >> system started >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService >> 15/09/08 14:35:24 INFO localizer.ResourceLocalizationService: per >> directory file limit = 8192 >> 15/09/08 14:35:24 INFO localizer.ResourceLocalizationService: >> usercache path : >> file:///tmp/hadoop-mapr/nm-local-dir/usercache_DEL_1441740924753 >> 15/09/08 14:35:24 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType >> for class >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker >> 15/09/08 14:35:24 WARN containermanager.AuxServices: The Auxilurary >> Service named 'mapreduce_shuffle' in the configuration is for class >> org.apache.hadoop.mapred.ShuffleHandler which has a name of >> 'httpshuffle'. Because these are not the same tools trying to send >> ServiceData and read Service Meta Data may have issues unless the >> refer to the name in the config. >> 15/09/08 14:35:24 INFO containermanager.AuxServices: Adding auxiliary >> service httpshuffle, "mapreduce_shuffle" >> 15/09/08 14:35:24 INFO monitor.ContainersMonitorImpl: Using >> ResourceCalculatorPlugin : >> org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@1a5b6f42 >> 15/09/08 14:35:24 INFO monitor.ContainersMonitorImpl: Using >> ResourceCalculatorProcessTree : null >> 15/09/08 14:35:24 INFO monitor.ContainersMonitorImpl: Physical memory >> check enabled: true >> 15/09/08 14:35:24 INFO monitor.ContainersMonitorImpl: Virtual memory >> check enabled: false >> 15/09/08 14:35:24 INFO nodemanager.NodeStatusUpdaterImpl: Initialized >> nodemanager for null: physical-memory=16384 virtual-memory=34407 >> virtual-cores=4 disks=4.0 >> 15/09/08 14:35:24 INFO ipc.CallQueueManager: Using callQueue class >> java.util.concurrent.LinkedBlockingQueue >> 15/09/08 14:35:24 INFO ipc.Server: Starting Socket Reader #1 for port >> 55449 >> 15/09/08 14:35:24 INFO pb.RpcServerFactoryPBImpl: Adding protocol >> org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server >> 15/09/08 14:35:24 INFO containermanager.ContainerManagerImpl: Blocking >> new container-requests as container manager rpc server is still >> starting. >> 15/09/08 14:35:24 INFO ipc.Server: IPC Server Responder: starting >> 15/09/08 14:35:24 INFO ipc.Server: IPC Server listener on 55449: starting >> 15/09/08 14:35:24 INFO security.NMContainerTokenSecretManager: >> Updating node address : hadoopmapr5.brewingintel.com:55449 >> 15/09/08 14:35:24 INFO ipc.CallQueueManager: Using callQueue class >> java.util.concurrent.LinkedBlockingQueue >> 15/09/08 14:35:24 INFO ipc.Server: Starting Socket Reader #1 for port 8040 >> 15/09/08 14:35:24 INFO pb.RpcServerFactoryPBImpl: Adding protocol >> org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB >> to the server >> 15/09/08 14:35:24 INFO ipc.Server: IPC Server Responder: starting >> 15/09/08 14:35:24 INFO ipc.Server: IPC Server listener on 8040: starting >> 15/09/08 14:35:24 INFO localizer.ResourceLocalizationService: >> Localizer started on port 8040 >> 15/09/08 14:35:24 INFO mapred.IndexCache: IndexCache created with max >> memory = 10485760 >> 15/09/08 14:35:24 INFO mapred.ShuffleHandler: httpshuffle listening on >> port 13562 >> 15/09/08 14:35:24 INFO containermanager.ContainerManagerImpl: >> ContainerManager started at hadoopmapr5/192.168.0.96:55449 >> 15/09/08 14:35:24 INFO containermanager.ContainerManagerImpl: >> ContainerManager bound to 0.0.0.0/0.0.0.0:0 >> 15/09/08 14:35:24 INFO webapp.WebServer: Instantiating NMWebApp at >> 0.0.0.0:8042 >> 15/09/08 14:35:24 INFO mortbay.log: Logging to >> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> org.mortbay.log.Slf4jLog >> 15/09/08 14:35:24 INFO http.HttpRequestLog: Http request log for >> http.requests.nodemanager is not defined >> 15/09/08 14:35:24 INFO http.HttpServer2: Added global filter 'safety' >> (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) >> 15/09/08 14:35:24 INFO http.HttpServer2: Added filter >> static_user_filter >> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) >> to context node >> 15/09/08 14:35:24 INFO http.HttpServer2: Added filter >> static_user_filter >> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) >> to context static >> 15/09/08 14:35:24 INFO http.HttpServer2: Added filter >> static_user_filter >> (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) >> to context logs >> 15/09/08 14:35:24 INFO http.HttpServer2: adding path spec: /node/* >> 15/09/08 14:35:24 INFO http.HttpServer2: adding path spec: /ws/* >> 15/09/08 14:35:24 INFO http.HttpServer2: Jetty bound to port 8042 >> 15/09/08 14:35:24 INFO mortbay.log: jetty-6.1.26 >> 15/09/08 14:35:24 INFO mortbay.log: Extract >> >> jar:file:/tmp/mesos/slaves/20150907-111332-1660987584-5050-8033-S3/frameworks/20150907-111332-1660987584-5050-8033-0003/executors/myriad_executor20150907-111332-1660987584-5050-8033-000320150907-111332-1660987584-5050-8033-O11824820150907-111332-1660987584-5050-8033-S3/runs/67cc8f37-b6d4-4018-a9b4-0071d020c9a5/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-common-2.7.0-mapr-1506.jar!/webapps/node >> to /tmp/Jetty_0_0_0_0_8042_node____19tj0x/webapp >> 15/09/08 14:35:25 INFO mortbay.log: Started >> HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:8042 >> 15/09/08 14:35:25 INFO webapp.WebApps: Web app /node started at 8042 >> 15/09/08 14:35:25 INFO webapp.WebApps: Registered webapp guice modules >> 15/09/08 14:35:25 INFO client.RMProxy: Connecting to ResourceManager >> at myriad.marathon.mesos/192.168.0.99:8031 >> 15/09/08 14:35:25 INFO nodemanager.NodeStatusUpdaterImpl: Sending out >> 0 NM container statuses: [] >> 15/09/08 14:35:25 INFO nodemanager.NodeStatusUpdaterImpl: Registering >> with RM using containers :[] >> 15/09/08 14:35:25 INFO security.NMContainerTokenSecretManager: Rolling >> master-key for container-tokens, got key with id 338249572 >> 15/09/08 14:35:25 INFO security.NMTokenSecretManagerInNM: Rolling >> master-key for container-tokens, got key with id -362725484 >> 15/09/08 14:35:25 INFO nodemanager.NodeStatusUpdaterImpl: Registered >> with ResourceManager as hadoopmapr5.brewingintel.com:55449 with total >> resource of <memory:16384, vCores:4, disks:4.0> >> 15/09/08 14:35:25 INFO nodemanager.NodeStatusUpdaterImpl: Notifying >> ContainerManager to unblock new container-requests >> >> >> Except of Myriad logs: >> >> /09/08 14:35:12 INFO handlers.ResourceOffersEventHandler: Received offers >> 1 >> 15/09/08 14:35:13 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:15 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:16 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:17 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:18 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:19 INFO handlers.StatusUpdateEventHandler: Status >> Update for task: value: >> "nm.medium.323f6664-11ca-477b-9e6e-41fb7547eacf" >> | state: TASK_FAILED >> 15/09/08 14:35:19 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:19 INFO scheduler.DownloadNMExecutorCLGenImpl: Using >> remote distribution >> 15/09/08 14:35:19 INFO scheduler.TaskFactory$NMTaskFactoryImpl: >> Getting Hadoop distribution >> from:maprfs:///mesos/myriad/hadoop-2.7.0.tar.gz >> 15/09/08 14:35:19 INFO scheduler.TaskFactory$NMTaskFactoryImpl: >> Getting config from:http://myriad.marathon.mesos:8088/conf >> 15/09/08 14:35:19 INFO scheduler.TaskFactory$NMTaskFactoryImpl: Slave >> will execute command:sudo tar -zxpf hadoop-2.7.0.tar.gz && sudo chown >> mapr . && cp conf hadoop-2.7.0/etc/hadoop/yarn-site.xml; export >> YARN_HOME=hadoop-2.7.0; sudo -E -u mapr -H env >> YARN_HOME="hadoop-2.7.0" >> YARN_NODEMANAGER_OPTS="-Dnodemanager.resource.io-spindles=4.0 >> -Dyarn.resourcemanager.hostname=myriad.marathon.mesos >> >> -Dyarn.nodemanager.container-executor.class=org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor >> -Dnodemanager.resource.cpu-vcores=4 >> -Dnodemanager.resource.memory-mb=16384 >> -Dmyriad.yarn.nodemanager.address=0.0.0.0:31000 >> -Dmyriad.yarn.nodemanager.localizer.address=0.0.0.0:31001 >> -Dmyriad.yarn.nodemanager.webapp.address=0.0.0.0:31002 >> -Dmyriad.mapreduce.shuffle.port=0.0.0.0:31003" $YARN_HOME/bin/yarn >> nodemanager >> 15/09/08 14:35:19 INFO handlers.ResourceOffersEventHandler: Launching >> task: nm.medium.323f6664-11ca-477b-9e6e-41fb7547eacf using offer: >> value: "20150907-111332-1660987584-5050-8033-O118248" >> >> 15/09/08 14:35:20 INFO handlers.ResourceOffersEventHandler: Received >> offers 2 >> 15/09/08 14:35:21 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:21 INFO util.AbstractLivelinessMonitor: >> Expired:hadoopmapr5.brewingintel.com:52878 Timed out after 2 secs >> 15/09/08 14:35:21 INFO rmnode.RMNodeImpl: Deactivating Node >> hadoopmapr5.brewingintel.com:52878 as it is now LOST >> 15/09/08 14:35:21 INFO rmnode.RMNodeImpl: >> hadoopmapr5.brewingintel.com:52878 Node Transitioned from RUNNING to >> LOST >> 15/09/08 14:35:21 INFO fair.FairScheduler: Removed node >> hadoopmapr5.brewingintel.com:52878 cluster capacity: <memory:0, >> vCores:0, disks:0.0> >> 15/09/08 14:35:22 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:23 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:25 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:25 INFO util.RackResolver: Resolved >> hadoopmapr5.brewingintel.com to /default-rack >> 15/09/08 14:35:25 INFO resourcemanager.ResourceTrackerService: >> NodeManager from node hadoopmapr5.brewingintel.com(cmPort: 55449 >> httpPort: 8042) registered with capability: <memory:16384, vCores:4, >> disks:4.0>, assigned nodeId hadoopmapr5.brewingintel.com:55449 >> 15/09/08 14:35:25 INFO rmnode.RMNodeImpl: >> hadoopmapr5.brewingintel.com:55449 Node Transitioned from NEW to >> RUNNING >> 15/09/08 14:35:25 INFO fair.FairScheduler: Added node >> hadoopmapr5.brewingintel.com:55449 cluster capacity: <memory:16384, >> vCores:4, disks:4.0> >> 15/09/08 14:35:26 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:27 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:28 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:30 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:31 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:32 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:33 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:35 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:36 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:37 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:38 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:40 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:41 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:42 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:43 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:45 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:46 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:47 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:48 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:50 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:51 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:52 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:53 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:55 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:35:56 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:57 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:35:58 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:00 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:36:01 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:02 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:03 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:05 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:36:06 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:07 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:08 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:10 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:36:11 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:12 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:13 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:15 INFO handlers.ResourceOffersEventHandler: Received >> offers 3 >> 15/09/08 14:36:16 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:17 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:18 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:19 INFO handlers.StatusUpdateEventHandler: Status >> Update for task: value: >> "nm.medium.323f6664-11ca-477b-9e6e-41fb7547eacf" >> | state: TASK_FAILED >> 15/09/08 14:36:19 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:19 INFO scheduler.DownloadNMExecutorCLGenImpl: Using >> remote distribution >> 15/09/08 14:36:19 INFO scheduler.TaskFactory$NMTaskFactoryImpl: >> Getting Hadoop distribution >> from:maprfs:///mesos/myriad/hadoop-2.7.0.tar.gz >> 15/09/08 14:36:19 INFO scheduler.TaskFactory$NMTaskFactoryImpl: >> Getting config from:http://myriad.marathon.mesos:8088/conf >> 15/09/08 14:36:19 INFO scheduler.TaskFactory$NMTaskFactoryImpl: Slave >> will execute command:sudo tar -zxpf hadoop-2.7.0.tar.gz && sudo chown >> mapr . && cp conf hadoop-2.7.0/etc/hadoop/yarn-site.xml; export >> YARN_HOME=hadoop-2.7.0; sudo -E -u mapr -H env >> YARN_HOME="hadoop-2.7.0" >> YARN_NODEMANAGER_OPTS="-Dnodemanager.resource.io-spindles=4.0 >> -Dyarn.resourcemanager.hostname=myriad.marathon.mesos >> >> -Dyarn.nodemanager.container-executor.class=org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor >> -Dnodemanager.resource.cpu-vcores=4 >> -Dnodemanager.resource.memory-mb=16384 >> -Dmyriad.yarn.nodemanager.address=0.0.0.0:31000 >> -Dmyriad.yarn.nodemanager.localizer.address=0.0.0.0:31001 >> -Dmyriad.yarn.nodemanager.webapp.address=0.0.0.0:31002 >> -Dmyriad.mapreduce.shuffle.port=0.0.0.0:31003" $YARN_HOME/bin/yarn >> nodemanager >> 15/09/08 14:36:19 INFO handlers.ResourceOffersEventHandler: Launching >> task: nm.medium.323f6664-11ca-477b-9e6e-41fb7547eacf using offer: >> value: "20150907-111332-1660987584-5050-8033-O118392" >> >> 15/09/08 14:36:20 INFO handlers.ResourceOffersEventHandler: Received >> offers 2 >> 15/09/08 14:36:20 INFO util.AbstractLivelinessMonitor: >> Expired:hadoopmapr5.brewingintel.com:55449 Timed out after 2 secs >> 15/09/08 14:36:20 INFO rmnode.RMNodeImpl: Deactivating Node >> hadoopmapr5.brewingintel.com:55449 as it is now LOST >> 15/09/08 14:36:20 INFO rmnode.RMNodeImpl: >> hadoopmapr5.brewingintel.com:55449 Node Transitioned from RUNNING to >> LOST >> 15/09/08 14:36:20 INFO fair.FairScheduler: Removed node >> hadoopmapr5.brewingintel.com:55449 cluster capacity: <memory:0, >> vCores:0, disks:0.0> >> 15/09/08 14:36:22 INFO handlers.ResourceOffersEventHandler: Received >> offers 2 >> 15/09/08 14:36:23 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:24 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:25 INFO handlers.ResourceOffersEventHandler: Received >> offers 2 >> 15/09/08 14:36:25 INFO util.RackResolver: Resolved >> hadoopmapr5.brewingintel.com to /default-rack >> 15/09/08 14:36:25 INFO resourcemanager.ResourceTrackerService: >> NodeManager from node hadoopmapr5.brewingintel.com(cmPort: 40378 >> httpPort: 8042) registered with capability: <memory:16384, vCores:4, >> disks:4.0>, assigned nodeId hadoopmapr5.brewingintel.com:40378 >> 15/09/08 14:36:25 INFO rmnode.RMNodeImpl: >> hadoopmapr5.brewingintel.com:40378 Node Transitioned from NEW to >> RUNNING >> 15/09/08 14:36:25 INFO fair.FairScheduler: Added node >> hadoopmapr5.brewingintel.com:40378 cluster capacity: <memory:16384, >> vCores:4, disks:4.0> >> 15/09/08 14:36:27 INFO handlers.ResourceOffersEventHandler: Received >> offers 2 >> 15/09/08 14:36:28 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:29 INFO handlers.ResourceOffersEventHandler: Received >> offers 1 >> 15/09/08 14:36:30 INFO handlers.ResourceOffersEventHandler: Received >> offers 2 >> >> >> > >
