Thanks Arun; I've filed https://issues.apache.org/jira/browse/YARN-511
On Thu, Mar 28, 2013 at 2:41 AM, Arun C Murthy <[email protected]> wrote: > That seems like a bug - we should at least throw a better error. > > On Mar 25, 2013, at 5:29 PM, Harsh J wrote: > > YARN does not seem to be checking for a fully qualified path when you > pass it yours and ends up breaking. The problem is easily reproducible > with the two transforming calls from ConverterUtils. > > Transform the jarPath to a fully qualified one like so, before using > it anywhere: > > Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR); > jarPath = fs.makeQualified(jarPath); > > And the problem should disappear. > > > On Mon, Mar 25, 2013 at 10:57 PM, blah blah <[email protected]> wrote: > > Hi > > > I am trying to create my own Application Master. I have followed this > > tutorial > > http://hadoop.apache.org/docs/r2.0.2-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html > > However I have problem with reading AM jar as resource at the container. > > Code for passing AM.jar > > > Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR); > > fs.copyFromLocalFile(new Path("/local/src/AM.jar"), jarPath); // VALIDATED > > jar is in HDFS under correct PATH > > FileStatus jarStatus = fs.getFileStatus(jarPath); > > LocalResource amJarRsrc = Records.newRecord(LocalResource.class); > > amJarRsrc.setType(LocalResourceType.FILE); > > amJarRsrc.setVisibility(LocalResourceVisibility.APPLICATION); > > amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(jarPath)); > > amJarRsrc.setTimestamp(jarStatus.getModificationTime()); > > amJarRsrc.setSize(jarStatus.getLen()); > > localResources.put("AppMaster.jar", amJarRsrc); > > amContainer.setLocalResources(localResources); > > > Error logs (nodeManager.log) > > > INFO > > org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: > > Application application_1364219323374_0016 transitioned from INITING to > > RUNNING > > INFO > > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: > > Got exception parsing AppMaster.jar and value resource {, port: -1, file: > > "/Working_HDFS_DIR/application_1364219323374_0016/AM.jar", }, size: 13940, > > timestamp: 1364230436600, type: FILE, visibility: APPLICATION, > > 2013-03-25 17:53:57,391 WARN > > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: > > Failed to parse resource-request > > java.net.URISyntaxException: Expected scheme name at index 0: > > :///Working_HDFS_DIR/application_1364219323374_0016/AM.jar > > at java.net.URI$Parser.fail(URI.java:2810) > > at java.net.URI$Parser.failExpecting(URI.java:2816) > > at java.net.URI$Parser.parse(URI.java:3008) > > at java.net.URI.<init>(URI.java:735) > > at > > org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:70) > > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46) > > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:501) > > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:472) > > at > > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:382) > > at > > org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299) > > at > > org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) > > at > > org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445) > > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:828) > > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:71) > > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:556) > > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:549) > > at > > org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:130) > > at > > org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77) > > at java.lang.Thread.run(Thread.java:662) > > > Where am I making the mistake? > > > regards > > tmp > > > > > -- > Harsh J > > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > -- Harsh J
