[
https://issues.apache.org/jira/browse/HADOOP-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835943#action_12835943
]
Tsz Wo (Nicholas), SZE commented on HADOOP-6564:
------------------------------------------------
The problem is that
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(Job job,
Path path) uses the default FileSystem, instead of the FileSystem specified in
the path.
{code}
//org.apache.hadoop.mapreduce.lib.input.FileInputFormat
public static void addInputPath(Job job,
Path path) throws IOException {
Configuration conf = job.getConfiguration();
FileSystem fs = FileSystem.get(conf);
path = path.makeQualified(fs); // In our case, path scheme is har:
// but fs scheme is hdfs://
...
}
{code}
> HarFileSystem cannot handle har:///
> -----------------------------------
>
> Key: HADOOP-6564
> URL: https://issues.apache.org/jira/browse/HADOOP-6564
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Mahadev konar
> Fix For: 0.22.0
>
>
> {noformat}
> -bash-3.1$ hadoop jar examples.jar wordcount -Dmapred.job.queue.name=${JOBQ}
> har:///user/tsz/t10_4.har/t10_4 t10_4_har_wc
> java.io.IOException: No FileSystem for scheme: namenode_hostname
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1375)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
> at
> org.apache.hadoop.fs.HarFileSystem.initialize(HarFileSystem.java:104)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:193)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:203)
> ...
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
