Please ignore below emails, looks like Yarn do need HDFS to download applications jar on nodemanager before they can run .
Thanks for the patience On 20 March 2014 18:05, Kapoor <[email protected]> wrote: > Is it because I am not running datanode therefore ResourceLocalizationService > is unable to download the jar ? > > > On 20 March 2014 17:42, Kapoor <[email protected]> wrote: > >> Hi, >> >> Got code latest code and build it. >> >> Below is the case. >> >> Setup of two machines each having same user and group used for YARN setup. >> >> Only YARN is running that is Resourcemanager and nodemanager >> >> when I ran DistributedShell example I get the below errors and >> application fails >> >> 2014-03-20 17:35:58,776 INFO SecurityLogger.org.apache.hadoop.ipc.Server: >> Auth successful for appattempt_1395316519303_0002_000001 (auth:SIMPLE) >> 2014-03-20 17:35:58,798 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: >> Start request for container_1395316519303_0002_01_000001 by user abhishek >> 2014-03-20 17:35:58,798 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: >> Creating a new application reference for app application_1395316519303_0002 >> 2014-03-20 17:35:58,798 INFO >> org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=abhishek >> IP=192.168.213.80 OPERATION=Start Container Request >> TARGET=ContainerManageImpl RESULT=SUCCESS >> APPID=application_1395316519303_0002 >> CONTAINERID=container_1395316519303_0002_01_000001 >> 2014-03-20 17:35:58,798 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: >> Application application_1395316519303_0002 transitioned from NEW to INITING >> 2014-03-20 17:35:58,799 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: >> Adding container_1395316519303_0002_01_000001 to application >> application_1395316519303_0002 >> 2014-03-20 17:35:58,799 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: >> Application application_1395316519303_0002 transitioned from INITING to >> RUNNING >> 2014-03-20 17:35:58,800 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: >> Container container_1395316519303_0002_01_000001 transitioned from NEW to >> LOCALIZING >> 2014-03-20 17:35:58,800 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got >> event CONTAINER_INIT for appId application_1395316519303_0002 >> 2014-03-20 17:35:58,800 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: >> Resource file:/home/abhishek/DistributedShell/2/AppMaster.jar transitioned >> from INIT to DOWNLOADING >> 2014-03-20 17:35:58,800 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: >> Resource file:/home/abhishek/DistributedShell/2/shellCommands transitioned >> from INIT to DOWNLOADING >> 2014-03-20 17:35:58,800 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: >> Created localizer for container_1395316519303_0002_01_000001 >> 2014-03-20 17:35:58,806 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: >> Writing credentials to the nmPrivate file >> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/nmPrivate/container_1395316519303_0002_01_000001.tokens. >> Credentials list: >> 2014-03-20 17:35:58,810 INFO >> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: >> Initializing user abhishek >> 2014-03-20 17:35:58,818 INFO >> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Copying >> from >> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/nmPrivate/container_1395316519303_0002_01_000001.tokens >> to >> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002/container_1395316519303_0002_01_000001.tokens >> 2014-03-20 17:35:58,818 INFO >> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: CWD set >> to >> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002 >> = file:/home/abhishek >> >> */hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002 >> 2014-03-20 17:35:58,842 WARN >> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException >> as:abhishek (auth:SIMPLE) cause:java.io.FileNotFoundException: File >> file:/home/abhishek/DistributedShell/2/AppMaster.jar does not exist >> 2014-03-20 17:35:58,843 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: >> DEBUG: FAILED { file:/home/abhishek/DistributedShell/2/AppMaster.jar, >> 1395316985000, FILE, null }, File >> file:/home/abhishek/DistributedShell/2/AppMaster.jar does not exist* >> 2014-03-20 17:35:58,844 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: >> Resource file:/home/abhishek/DistributedShell/2/AppMaster.jar transitioned >> from* DOWNLOADING to FAILED* >> 2014-03-20 17:35:58,844 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: >> Container container_1395316519303_0002_01_000001 transitioned from >> LOCALIZING to LOCALIZATION_FAILED >> 2014-03-20 17:35:58,844 INFO >> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: >> Container container_1395316519303_0002_01_000001 sent RELEASE event on a >> resource request { file:/home/abhishek/DistributedShell/2/AppMaster.jar, >> 1395316985000, FILE, null } not present in cache. >> >> >> Looks like ResourceManager machine extracts DistributedShell jar and >> create home/abhishek/DistributedShell/ locally >> But Node manager machine does not get/create the folder structure and >> download the jar >> * . * >> >> Any pointer to help will be great. >> >> >> >> >> -- >> *Thanks and Regards* >> *Abhishek Kapoor* >> >> *LinkedIn : in.linkedin.com/in/abhishekkapoorbigdata/ >> <http://in.linkedin.com/in/abhishekkapoorbigdata/>* >> *Twitter: @kapoorSunny* >> > > > > -- > *Thanks and Regards* > *Abhishek Kapoor* > > *LinkedIn : in.linkedin.com/in/abhishekkapoorbigdata/ > <http://in.linkedin.com/in/abhishekkapoorbigdata/>* > *Twitter: @kapoorSunny* > -- *Thanks and Regards* *Abhishek Kapoor* *LinkedIn : in.linkedin.com/in/abhishekkapoorbigdata/ <http://in.linkedin.com/in/abhishekkapoorbigdata/>* *Twitter: @kapoorSunny*
