[
https://issues.apache.org/jira/browse/HADOOP-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429459#comment-13429459
]
Karthik Kambatla commented on HADOOP-8649:
------------------------------------------
Hi Daryn,
The trunk code seems to be the roughly the same as branch-1. So, I believe both
have the bug.
Let me illustrate the issue that I see through some psuedo-code. Let me know if
I am missing something here.
{code}
DistributedFileSystem dfs = ... // some initialization of distributed file
system.
ChecksumFileSystem cfs = new ChecksumFileSystem(dfs); // cfs.fs is set to dfs.
Path randomPath = new Path("random-path"); // some path with 'random' in it.
PathFilter randomFilter = new PathFilter() {
boolean accept(Path file) {return !file.toString().contains("random");}
};
FileStatus[] listWithoutFilter = cfs.listStatus(randomPath); // in turn calls
dfs.listStatus(randomPath, ChecksumFileSystem.DEFAULT_FILTER)
FileStatus[] listWithFilter = cfs.listStatus(randomPath, randomFilter); // in
turn calls dfs.listStatus(randomPath, randomFilter)
{code}
dfs.listStatus(Path, PathFilter) calls FileSystem.listStatus(Path, PathFilter),
which first calls dfs.listStatus(path) and then applies PathFilter. Hence,
while checksum filter is used in the first cfs.listStatus, it is not used in
the second call to listStatus().
> ChecksumFileSystem should have an overriding implementation of
> listStatus(Path, PathFilter)
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-8649
> URL: https://issues.apache.org/jira/browse/HADOOP-8649
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: HADOOP-8649_branch1.patch, HADOOP-8649_branch1.patch_v2,
> HADOOP-8649_branch1.patch_v3
>
>
> Currently, ChecksumFileSystem implements only listStatus(Path). The other
> form of listStatus(Path, PathFilter) is implemented by parent class
> FileSystem, and hence doesn't filter out check-sum files.
> The implementation should use a composite filter of passed Filter and the
> Checksum filter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
