[
https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963595#comment-13963595
]
Gera Shegalov commented on HDFS-6143:
-------------------------------------
[~szetszwo], it's easy to understand the issue if your review the unit test
modifications in the patch. As for our production use case, that works fine on
the LocalFileSystem and HDFS, please review splittable Lzo in elephant bird.
Hadoop-lzo's {{LzoIndex.readIndex}} reads the index file and exploits the fact
the FS should [error out opening a non-existing
file|https://github.com/twitter/hadoop-lzo/blob/master/src/main/java/com/hadoop/compression/lzo/LzoIndex.java#L176],
instead of using an RPC to get the file status directly or via {{exists}}.
> WebHdfsFileSystem open should throw FileNotFoundException for non-existing
> paths
> --------------------------------------------------------------------------------
>
> Key: HDFS-6143
> URL: https://issues.apache.org/jira/browse/HDFS-6143
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.3.0
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Priority: Blocker
> Fix For: 2.5.0
>
> Attachments: HDFS-6143-branch-2.4.0.v01.patch,
> HDFS-6143-trunk-after-HDFS-5570.v01.patch,
> HDFS-6143-trunk-after-HDFS-5570.v02.patch, HDFS-6143.v01.patch,
> HDFS-6143.v02.patch, HDFS-6143.v03.patch, HDFS-6143.v04.patch,
> HDFS-6143.v04.patch, HDFS-6143.v05.patch, HDFS-6143.v06.patch
>
>
> WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles
> non-existing paths.
> - 'open', does not really open anything, i.e., it does not contact the
> server, and therefore cannot discover FileNotFound, it's deferred until next
> read. It's counterintuitive and not how local FS or HDFS work. In POSIX you
> get ENOENT on open.
> [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java]
> is an example of the code that's broken because of this.
> - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST
> instead of SC_NOT_FOUND for non-exitsing paths
--
This message was sent by Atlassian JIRA
(v6.2#6252)
