[
https://issues.apache.org/jira/browse/HADOOP-10003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13782256#comment-13782256
]
Jason Dere commented on HADOOP-10003:
-------------------------------------
So I'm not a bit unsure as to why the test case is failing in the HadoopQA -
I've run this locally on both MacOS and CentOS VM, and the test passes for me.
If HarFileSystem doesn't override listLocatedStatus, the new unit test errors
out with the error shown below. It looks like
FilterFileSystem.listLocatedStatus() simply pushing the call down the its
underlying filesystem, but since this is a hadoop archive the underlying
filesystem won't know about the files in the archive, it needs to look at the
index/data files in the archive to be able to get information about the files
in the archive. This fix to HarFileSystem originally was in MAPREDUCE-1981
(patch v5), but seems to be missing in subsequent versions of the patch.
Running org.apache.hadoop.fs.TestHarFileSystemBasics
Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.869 sec <<<
FAILURE! - in org.apache.hadoop.fs.TestHarFileSystemBasics
testListLocatedStatus(org.apache.hadoop.fs.TestHarFileSystemBasics) Time
elapsed: 0.403 sec <<< ERROR!
java.io.FileNotFoundException: File dir1 does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:386)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1482)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1522)
at org.apache.hadoop.fs.FileSystem$4.<init>(FileSystem.java:1678)
at
org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1677)
at
org.apache.hadoop.fs.ChecksumFileSystem.listLocatedStatus(ChecksumFileSystem.java:579)
at
org.apache.hadoop.fs.FilterFileSystem.listLocatedStatus(FilterFileSystem.java:235)
at
org.apache.hadoop.fs.TestHarFileSystemBasics.testListLocatedStatus(TestHarFileSystemBasics.java:255)
> HarFileSystem.listLocatedStatus() fails
> ---------------------------------------
>
> Key: HADOOP-10003
> URL: https://issues.apache.org/jira/browse/HADOOP-10003
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.1.1-beta
> Reporter: Jason Dere
> Attachments: HADOOP-10003.1.patch, HADOOP-10003.2.patch,
> HADOOP-10003.3.patch
>
>
> It looks like HarFileSystem.listLocatedStatus() doesn't work properly because
> it is inheriting FilterFileSystem's implementation. This is causing archive
> unit tests to fail in Hive when using hadoop 2.1.1.
> If HarFileSystem overrides listLocatedStatus() to use FileSystem's
> implementation, the Hive unit tests pass.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
