[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-7352: --- Release Note: Change FileSystem#listStatus contract to never return null. Local filesystems prior to 3.0.0 returned null upon accesserror. It is considered erroneous. We should expect FileSystem#listStatus to throw IOException upon access error. > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.6.0 >Reporter: Matt Foley >Assignee: John Zhuge > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch, > HADOOP-7352.003.patch, HADOOP-7352.004.patch, HADOOP-7352.005.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HADOOP-7352: -- Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.6.0 >Reporter: Matt Foley >Assignee: John Zhuge > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch, > HADOOP-7352.003.patch, HADOOP-7352.004.patch, HADOOP-7352.005.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HADOOP-7352: -- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 Status: Resolved (was: Patch Available) Committed this to trunk. Thanks [~jzhuge] for the patch, [~ste...@apache.org] and all for the review and discussions! > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.6.0 >Reporter: Matt Foley >Assignee: John Zhuge > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch, > HADOOP-7352.003.patch, HADOOP-7352.004.patch, HADOOP-7352.005.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[
https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-7352:
---
Attachment: HADOOP-7352.005.patch
Patch 005:
* Fix javadoc for {{FileSystem#listStatus}}
> FileSystem#listStatus should throw IOE upon access error
>
>
> Key: HADOOP-7352
> URL: https://issues.apache.org/jira/browse/HADOOP-7352
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 2.6.0
>Reporter: Matt Foley
>Assignee: John Zhuge
> Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch,
> HADOOP-7352.003.patch, HADOOP-7352.004.patch, HADOOP-7352.005.patch
>
>
> In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should
> throw FileNotFoundException instead of returning null, when the target
> directory did not exist.
> However, in LocalFileSystem implementation today, FileSystem::listStatus
> still may return null, when the target directory exists but does not grant
> read permission. This causes NPE in many callers, for all the reasons cited
> in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for
> examples.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-7352: --- Attachment: HADOOP-7352.004.patch Patch 004: * Rebase * Fix 2 nits > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.6.0 >Reporter: Matt Foley >Assignee: John Zhuge > Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch, > HADOOP-7352.003.patch, HADOOP-7352.004.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-7352: --- Attachment: HADOOP-7352.003.patch Patch 003: * Fix checkstyle and whilespace > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.6.0 >Reporter: Matt Foley >Assignee: John Zhuge > Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch, > HADOOP-7352.003.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-7352: --- Status: Patch Available (was: In Progress) > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.6.0 >Reporter: Matt Foley >Assignee: John Zhuge > Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[
https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-7352:
---
Attachment: HADOOP-7352.002.patch
Patch 002:
* For directory, {{RawLocalFileSystem#listStatus}} calls {{FileUtil#list}} that
throws IOE upon access error
* Add unit test
{{FSMainOperationsBaseTest#testGlobStatusThrowsExceptionForUnreadableDir}}
* Add unit test {{TestPathData#testGlobThrowsExceptionForUnreadableDir}}
* Update {{filesystem.md}}
* Pass tests: TestFSMainOperationsLocalFileSystem, TestFSMainOperationsWebHdfs,
TestFSMainOperationsWebHdfs, and TestPathData.
> FileSystem#listStatus should throw IOE upon access error
>
>
> Key: HADOOP-7352
> URL: https://issues.apache.org/jira/browse/HADOOP-7352
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 2.6.0
>Reporter: Matt Foley
>Assignee: John Zhuge
> Attachments: HADOOP-7352.001.patch, HADOOP-7352.002.patch
>
>
> In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should
> throw FileNotFoundException instead of returning null, when the target
> directory did not exist.
> However, in LocalFileSystem implementation today, FileSystem::listStatus
> still may return null, when the target directory exists but does not grant
> read permission. This causes NPE in many callers, for all the reasons cited
> in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for
> examples.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-7352: --- Affects Version/s: 2.6.0 Target Version/s: 3.0.0-alpha2 Status: In Progress (was: Patch Available) > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.6.0 >Reporter: Matt Foley >Assignee: John Zhuge > Attachments: HADOOP-7352.001.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-7352: --- Status: Patch Available (was: In Progress) > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Reporter: Matt Foley >Assignee: John Zhuge > Attachments: HADOOP-7352.001.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[
https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-7352:
---
Attachment: HADOOP-7352.001.patch
Patch 001:
* {{RawLocalFileSystem#listStatus}} throws IOE upon access error. Leverage
{{FileUtil#list}}.
* Update {{filesystem.md}}
* Pass unit test {{TestFSMainOperationsLocalFileSystem}} and
{{TestFSMainOperationsWebHdfs}}
[~xiaochen] This patches follows {{HADOOP-13191.004.patch}}, addressing all
your review comments.
> FileSystem#listStatus should throw IOE upon access error
>
>
> Key: HADOOP-7352
> URL: https://issues.apache.org/jira/browse/HADOOP-7352
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Reporter: Matt Foley
>Assignee: John Zhuge
> Attachments: HADOOP-7352.001.patch
>
>
> In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should
> throw FileNotFoundException instead of returning null, when the target
> directory did not exist.
> However, in LocalFileSystem implementation today, FileSystem::listStatus
> still may return null, when the target directory exists but does not grant
> read permission. This causes NPE in many callers, for all the reasons cited
> in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for
> examples.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-7352) FileSystem#listStatus should throw IOE upon access error
[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-7352: --- Summary: FileSystem#listStatus should throw IOE upon access error (was: FileSystem#listStatus should throw IOException upon access error) > FileSystem#listStatus should throw IOE upon access error > > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Reporter: Matt Foley >Assignee: John Zhuge > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
