[
https://issues.apache.org/jira/browse/MAPREDUCE-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated MAPREDUCE-1522:
----------------------------------------------
Attachment: m1522_20100224.patch
Amareshwari, thanks for the review.
m1522_20100224.patch: incorporated both review comments.
Without the changes in FileInputFormat, it will fail on the new test as
following.
{noformat}
defaultfs.getUri() = s3://abc:x...@hostname
original = file:/foo
results = [file://abc:x...@hostname/foo]
------------- ---------------- ---------------
Testcase: testAddInputPath took 0.704 sec
FAILED
expected:<file:/foo> but was:<file://abc:x...@hostname/foo>
junit.framework.AssertionFailedError: expected:<file:/foo> but
was:<file://abc:x...@hostname/foo>
at
org.apache.hadoop.mapreduce.lib.input.TestFileInputFormat.testAddInputPath(TestFileInputFormat.java:54)
{noformat}
> FileInputFormat may change the file system of an input path
> -----------------------------------------------------------
>
> Key: MAPREDUCE-1522
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1522
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Tsz Wo (Nicholas), SZE
> Attachments: m1522_20100224.patch, m1522_20100224.patch,
> m1522_20100224_0.20.patch, m1552_20100223_0.20.patch
>
>
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(Job job,
> Path path) uses the default FileSystem but not the FileSystem specified in
> the path.
> {code}
> //org.apache.hadoop.mapreduce.lib.input.FileInputFormat
> public static void addInputPath(Job job,
> Path path) throws IOException {
> Configuration conf = job.getConfiguration();
> FileSystem fs = FileSystem.get(conf);
> path = path.makeQualified(fs); // the original FileSystem is lost.
> ...
> }
> {code}
> There is a similar problem in FileInputFormat.setInputPaths(..).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
