[
https://issues.apache.org/jira/browse/HADOOP-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jothi Padmanabhan updated HADOOP-3597:
--------------------------------------
Attachment: hadoop-3597.patch
Here is a patch that fixes the above issue.
Note that, even with this patch, it is not possible to use different hars for
input and output for the SortValidator as the Cache that stores the different
file systems is based only on the scheme, authority and user information and
does not use the entire har name. So,
bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput
har:///user/jothipn/foo.har/user/jothipn/input -sortOutput ouput
-->works
bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput input
-sortOutput har:///user/jothipn/foo1.har/user/jothipn/output -->works
bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput
har:///user/jothipn/foo.har/user/jothipn/input -sortOutput
har:///user/jothipn/foo1.har/user/jothipn/output -->does not work.
> SortValidator always uses the default file system irrespective of the actual
> input
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-3597
> URL: https://issues.apache.org/jira/browse/HADOOP-3597
> Project: Hadoop Core
> Issue Type: Bug
> Components: test
> Reporter: Jothi Padmanabhan
> Attachments: hadoop-3597.patch
>
>
> In SortValidator, the underlying file system should be obtained from the
> given inputs (sortInput and sortOutput). It is currently assumed to be the
> default (HDFS) always.
> So, the following usage does not work
> bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput
> har:///user/jothipn/foo.har/user/jothipn/input -sortOutput output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
