Is it because I am not running datanode therefore ResourceLocalizationService is unable to download the jar ?
On 20 March 2014 17:42, Kapoor <[email protected]> wrote: > Hi, > > Got code latest code and build it. > > Below is the case. > > Setup of two machines each having same user and group used for YARN setup. > > Only YARN is running that is Resourcemanager and nodemanager > > when I ran DistributedShell example I get the below errors and application > fails > > 2014-03-20 17:35:58,776 INFO SecurityLogger.org.apache.hadoop.ipc.Server: > Auth successful for appattempt_1395316519303_0002_000001 (auth:SIMPLE) > 2014-03-20 17:35:58,798 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: > Start request for container_1395316519303_0002_01_000001 by user abhishek > 2014-03-20 17:35:58,798 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: > Creating a new application reference for app application_1395316519303_0002 > 2014-03-20 17:35:58,798 INFO > org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=abhishek > IP=192.168.213.80 OPERATION=Start Container Request > TARGET=ContainerManageImpl RESULT=SUCCESS > APPID=application_1395316519303_0002 > CONTAINERID=container_1395316519303_0002_01_000001 > 2014-03-20 17:35:58,798 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: > Application application_1395316519303_0002 transitioned from NEW to INITING > 2014-03-20 17:35:58,799 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: > Adding container_1395316519303_0002_01_000001 to application > application_1395316519303_0002 > 2014-03-20 17:35:58,799 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: > Application application_1395316519303_0002 transitioned from INITING to > RUNNING > 2014-03-20 17:35:58,800 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: > Container container_1395316519303_0002_01_000001 transitioned from NEW to > LOCALIZING > 2014-03-20 17:35:58,800 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got > event CONTAINER_INIT for appId application_1395316519303_0002 > 2014-03-20 17:35:58,800 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: > Resource file:/home/abhishek/DistributedShell/2/AppMaster.jar transitioned > from INIT to DOWNLOADING > 2014-03-20 17:35:58,800 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: > Resource file:/home/abhishek/DistributedShell/2/shellCommands transitioned > from INIT to DOWNLOADING > 2014-03-20 17:35:58,800 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Created localizer for container_1395316519303_0002_01_000001 > 2014-03-20 17:35:58,806 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Writing credentials to the nmPrivate file > /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/nmPrivate/container_1395316519303_0002_01_000001.tokens. > Credentials list: > 2014-03-20 17:35:58,810 INFO > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: > Initializing user abhishek > 2014-03-20 17:35:58,818 INFO > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Copying > from > /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/nmPrivate/container_1395316519303_0002_01_000001.tokens > to > /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002/container_1395316519303_0002_01_000001.tokens > 2014-03-20 17:35:58,818 INFO > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: CWD set > to > /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002 > = file:/home/abhishek > > */hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002 > 2014-03-20 17:35:58,842 WARN > org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException > as:abhishek (auth:SIMPLE) cause:java.io.FileNotFoundException: File > file:/home/abhishek/DistributedShell/2/AppMaster.jar does not exist > 2014-03-20 17:35:58,843 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > DEBUG: FAILED { file:/home/abhishek/DistributedShell/2/AppMaster.jar, > 1395316985000, FILE, null }, File > file:/home/abhishek/DistributedShell/2/AppMaster.jar does not exist* > 2014-03-20 17:35:58,844 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: > Resource file:/home/abhishek/DistributedShell/2/AppMaster.jar transitioned > from* DOWNLOADING to FAILED* > 2014-03-20 17:35:58,844 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: > Container container_1395316519303_0002_01_000001 transitioned from > LOCALIZING to LOCALIZATION_FAILED > 2014-03-20 17:35:58,844 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl: > Container container_1395316519303_0002_01_000001 sent RELEASE event on a > resource request { file:/home/abhishek/DistributedShell/2/AppMaster.jar, > 1395316985000, FILE, null } not present in cache. > > > Looks like ResourceManager machine extracts DistributedShell jar and > create home/abhishek/DistributedShell/ locally > But Node manager machine does not get/create the folder structure and > download the jar > * . * > > Any pointer to help will be great. > > > > > -- > *Thanks and Regards* > *Abhishek Kapoor* > > *LinkedIn : in.linkedin.com/in/abhishekkapoorbigdata/ > <http://in.linkedin.com/in/abhishekkapoorbigdata/>* > *Twitter: @kapoorSunny* > -- *Thanks and Regards* *Abhishek Kapoor* *LinkedIn : in.linkedin.com/in/abhishekkapoorbigdata/ <http://in.linkedin.com/in/abhishekkapoorbigdata/>* *Twitter: @kapoorSunny*
