Hi Silvan, Are you trying to run MapReduce? You could look at createApplicationSubmissionContext and createApplicationResource in YarnRunner.java. Have you overridden the resolvePath function in your filesystem implementation?
-Varun From: Silvan Kaiser Reply-To: "[email protected]<mailto:[email protected]>" Date: Friday, May 8, 2015 at 3:52 PM To: "[email protected]<mailto:[email protected]>" Subject: URI missing scheme and authority in job start with new FileSystem implementation Hello! I am running a new FileSystem implementation, testing it with a simple standalone operation from the Hadoop "Single Node Setup" page. Staging example jobs works alright but upon scheduling the container management fails because of an URI without scheme (and authority). NodeManager log snippet: ... 2015-05-08 15:18:15,331 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Got exception parsing jobSubmitDir/job.splitmetainfo and value resource { port: -1 file: "/tmp/hadoop-yarn/staging/kaisers/.staging/job_1430315033077_0086/job.splitmetainfo" } size: 771 timestamp: 1431091092000 type: FILE visibility: APPLICATION 2015-05-08 15:18:15,331 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Failed to parse resource-request java.net.URISyntaxException: Expected scheme name at index 0: :///tmp/hadoop-yarn/staging/kaisers/.staging/job_1430315033077_0086/job.splitmetainfo ... The path element of the URI is correct, it simply is missing it's scheme and authority but i cannot find out why so far. Please note that the URIs used during staging are correct and contain scheme & authority correctly. This means the incomplete URI is created during job creation. My guess so far: this is either because I am missing some configuration for the Nodemanager (therefore he does not know about my FileSystem implementation and throws the unknown scheme & authority out the window) or that the FileSystem implementation has an error / did not override a method that requires adoption. Can someone please give me a hint where the URIs used by the NodeManager are constructed prior to scheduling? This would allow me to trace back where the incomplete URI is coming from... Best regards Silvan -- Quobyte GmbH Boyenstr. 41 - 10115 Berlin-Mitte - Germany +49-30-814 591 800 - www.quobyte.com <http://www.quobyte.com/><http://www.quobyte.com/> Amtsgericht Berlin-Charlottenburg, HRB 149012B Management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender -- Quobyte GmbH Hardenbergplatz 2 - 10623 Berlin - Germany +49-30-814 591 800 - www.quobyte.com <http://www.quobyte.com/>Amtsgericht Berlin-Charlottenburg, HRB 149012B management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender
