Hi,
I am trying to submit a remote mapreduce job in Yarn with the
configuration below, but I get the error |No FileSystem for scheme:
hdfs|. The error is so also below. Why this is happening?
| String host = "192.168.56.100";
Configuration conf = job.getConfiguration();
// this should be like defined in your yarn-site.xml
conf.set("yarn.resourcemanager.address", host + ":50001");
// framework is now "yarn", should be defined like this in
mapred-site.xm
conf.set("mapreduce.framework.name", "yarn");
// like defined in hdfs-site.xml
conf.set("fs.defaultFS", "hdfs://" + host + ":9000");
for (Path inputPath : inputPaths)
FileInputFormat.addInputPath(job, new Path(conf.get("fs.defaultFS") +
"/" + inputPath.toString()));
FileOutputFormat.setOutputPath(job, new Path(conf.get("fs.defaultFS") +
"/" + tempPath));
job.waitForCompletion(true);
|
|Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:518)
at
org.apache.hadoop.mapred.examples.JobExecution.submit(JobExecution.java:169)
at org.apache.hadoop.mapred.examples.WordCount.main(WordCount.java:101)
|
Thanks,