Hi Bigtop, I have used puppet to deploy Hadoop onto a master and three slaves, however on the slaves, nodemanager will not start.
The errors found in /var/log/hadoop-yarn/yarn-yarn-nodemanager-slave1.log are here: 2014-07-26 01:24:22,370 ERROR > org.apache.hadoop.yarn.service.CompositeService: Error starting services > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl > org.apache.hadoop.yarn.YarnException: Failed to check for existence of > remoteLogDir [/var/log/hadoop-yarn/apps] > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:179) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.start(LogAggregationService.java:132) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.start(ContainerManagerImpl.java:248) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:199) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:322) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:359) > 2014-07-26 01:24:22,372 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService: > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService > waiting for pending aggregation during exit > 2014-07-26 01:24:22,372 INFO > org.apache.hadoop.yarn.service.AbstractService: Service:Dispatcher is > stopped. > 2014-07-26 01:24:22,372 WARN > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl > is interrupted. Exiting. > 2014-07-26 01:24:22,372 INFO > org.apache.hadoop.yarn.service.AbstractService: Service:containers-monitor > is stopped. > 2014-07-26 01:24:22,379 INFO > org.apache.hadoop.yarn.service.AbstractService: Service:httpshuffle is > stopped. > 2014-07-26 01:24:22,379 INFO > org.apache.hadoop.yarn.service.AbstractService: > Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices > is stopped. > 2014-07-26 01:24:22,379 INFO > org.apache.hadoop.yarn.service.AbstractService: Service:containers-launcher > is stopped. > 2014-07-26 01:24:22,379 INFO org.apache.hadoop.ipc.Server: Stopping server > on 8040 > 2014-07-26 01:24:22,380 INFO org.apache.hadoop.ipc.Server: Stopping IPC > Server listener on 8040 > 2014-07-26 01:24:22,380 INFO org.apache.hadoop.ipc.Server: Stopping IPC > Server Responder > 2014-07-26 01:24:22,380 INFO > org.apache.hadoop.yarn.service.AbstractService: > Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker > is stopped. > 2014-07-26 01:24:22,380 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Public cache exiting > 2014-07-26 01:24:22,380 INFO > org.apache.hadoop.yarn.service.AbstractService: > Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService > is stopped. > 2014-07-26 01:24:22,380 ERROR > org.apache.hadoop.yarn.service.CompositeService: Error starting services > org.apache.hadoop.yarn.server.nodemanager.NodeManager > org.apache.hadoop.yarn.YarnException: Failed to Start > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.start(ContainerManagerImpl.java:248) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:199) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:322) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:359) > Caused by: org.apache.hadoop.yarn.YarnException: Failed to check for > existence of remoteLogDir [/var/log/hadoop-yarn/apps] > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:179) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.start(LogAggregationService.java:132) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > ... 5 more > 2014-07-26 01:24:22,380 INFO org.apache.hadoop.ipc.Server: Stopping server > on 42208 > 2014-07-26 01:24:22,380 INFO org.apache.hadoop.ipc.Server: Stopping IPC > Server listener on 42208 > 2014-07-26 01:24:22,381 INFO org.apache.hadoop.ipc.Server: Stopping IPC > Server Responder > 2014-07-26 01:24:22,380 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService: > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService > waiting for pending aggregation during exit > 2014-07-26 01:24:22,381 INFO org.apache.hadoop.ipc.Server: Stopping server > on 8040 > 2014-07-26 01:24:22,381 INFO > org.apache.hadoop.yarn.service.AbstractService: > Service:org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl > is stopped. > 2014-07-26 01:24:22,381 INFO > org.apache.hadoop.yarn.service.AbstractService: > Service:org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService > is stopped. > 2014-07-26 01:24:22,381 INFO > org.apache.hadoop.yarn.service.AbstractService: > Service:org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService > is stopped. > 2014-07-26 01:24:22,381 INFO > org.apache.hadoop.yarn.service.AbstractService: > Service:org.apache.hadoop.yarn.server.nodemanager.DeletionService is > stopped. > 2014-07-26 01:24:22,381 FATAL > org.apache.hadoop.yarn.server.nodemanager.NodeManager: Error starting > NodeManager > org.apache.hadoop.yarn.YarnException: Failed to Start > org.apache.hadoop.yarn.server.nodemanager.NodeManager > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:199) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:322) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:359) > Caused by: org.apache.hadoop.yarn.YarnException: Failed to Start > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.start(ContainerManagerImpl.java:248) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > ... 3 more > Caused by: org.apache.hadoop.yarn.YarnException: Failed to check for > existence of remoteLogDir [/var/log/hadoop-yarn/apps] > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:179) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.start(LogAggregationService.java:132) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > ... 5 more > 2014-07-26 01:24:22,382 INFO org.apache.hadoop.ipc.Server: Stopping server > on 42208 > 2014-07-26 01:24:22,382 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService: > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService > waiting for pending aggregation during exit > 2014-07-26 01:24:22,382 INFO org.apache.hadoop.ipc.Server: Stopping server > on 8040 > 2014-07-26 01:24:22,382 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NodeManager > metrics system... > 2014-07-26 01:24:22,383 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics > system stopped. > 2014-07-26 01:24:22,383 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics > system shutdown complete. > 2014-07-26 01:24:22,385 INFO > org.apache.hadoop.yarn.server.nodemanager.NodeManager: SHUTDOWN_MSG: > /************************************************************ > SHUTDOWN_MSG: Shutting down NodeManager at slave1/127.0.0.1 > ************************************************************/ What can I do to fix this? -David Fryer
