[jira] [Commented] (HDFS-6143) (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13961203#comment-13961203 ] Haohui Mai commented on HDFS-6143: -- Can you please separate the patch for webhdfs and hftp? Hftp has been deprecated thus this part does not need necessarily to be a blocker. Otherwise it becomes difficult to review. Thanks. > (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for > non-existing paths > --- > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch, > HDFS-6143.v03.patch, HDFS-6143.v04.patch, HDFS-6143.v04.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6143) (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957402#comment-13957402 ] Gera Shegalov commented on HDFS-6143: - The exception in https://builds.apache.org/job/PreCommit-HDFS-Build/6561//testReport/ is reported in HDFS-6183. > (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for > non-existing paths > --- > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch, > HDFS-6143.v03.patch, HDFS-6143.v04.patch, HDFS-6143.v04.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6143) (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths
[
https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955627#comment-13955627
]
Hadoop QA commented on HDFS-6143:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637882/HDFS-6143.v04.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6564//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6564//console
This message is automatically generated.
> (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for
> non-existing paths
> ---
>
> Key: HDFS-6143
> URL: https://issues.apache.org/jira/browse/HDFS-6143
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch,
> HDFS-6143.v03.patch, HDFS-6143.v04.patch, HDFS-6143.v04.patch
>
>
> WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles
> non-existing paths.
> - 'open', does not really open anything, i.e., it does not contact the
> server, and therefore cannot discover FileNotFound, it's deferred until next
> read. It's counterintuitive and not how local FS or HDFS work. In POSIX you
> get ENOENT on open.
> [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java]
> is an example of the code that's broken because of this.
> - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST
> instead of SC_NOT_FOUND for non-exitsing paths
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6143) (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths
[
https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955077#comment-13955077
]
Hadoop QA commented on HDFS-6143:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637773/HDFS-6143.v04.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.web.TestWebHDFS
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6561//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6561//console
This message is automatically generated.
> (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for
> non-existing paths
> ---
>
> Key: HDFS-6143
> URL: https://issues.apache.org/jira/browse/HDFS-6143
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch,
> HDFS-6143.v03.patch, HDFS-6143.v04.patch
>
>
> WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles
> non-existing paths.
> - 'open', does not really open anything, i.e., it does not contact the
> server, and therefore cannot discover FileNotFound, it's deferred until next
> read. It's counterintuitive and not how local FS or HDFS work. In POSIX you
> get ENOENT on open.
> [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java]
> is an example of the code that's broken because of this.
> - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST
> instead of SC_NOT_FOUND for non-exitsing paths
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6143) (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955014#comment-13955014 ] Arun C Murthy commented on HDFS-6143: - [~jira.shegalov] Saw this too late - I'm moving this to 2.4.1, let's try get this in asap. I have rc0 going out, we can re-target 2.4.0 if I need to spin another rc. Thanks. > (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for > non-existing paths > --- > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch, > HDFS-6143.v03.patch, HDFS-6143.v04.patch > > > HftpFileSystem.open incorrectly handles non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6143) (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths
[
https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955005#comment-13955005
]
Hadoop QA commented on HDFS-6143:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637767/HDFS-6143.v03.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6559//console
This message is automatically generated.
> (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for
> non-existing paths
> ---
>
> Key: HDFS-6143
> URL: https://issues.apache.org/jira/browse/HDFS-6143
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch,
> HDFS-6143.v03.patch
>
>
> HftpFileSystem.open incorrectly handles non-existing paths.
> - 'open', does not really open anything, i.e., it does not contact the
> server, and therefore cannot discover FileNotFound, it's deferred until next
> read. It's counterintuitive and not how local FS or HDFS work. In POSIX you
> get ENOENT on open.
> [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java]
> is an example of the code that's broken because of this.
> - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST
> instead of SC_NOT_FOUND for non-exitsing paths
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6143) (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13954844#comment-13954844 ] Gera Shegalov commented on HDFS-6143: - WebHdfsFileSystem has the same problem. I'll update the patch to reflect. > (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for > non-existing paths > --- > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Critical > Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch > > > HftpFileSystem.open incorrectly handles non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
