[ https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472754#comment-15472754 ]
John Zhuge commented on HADOOP-7352: ------------------------------------ Move review comments from HADOOP-13191 to here. Steve commented: bq. Be aware that the FS shell expects globStatus to return null in certain conditions: someone needs to look at all uses of this call and make sure that it isn't being used John commented: Went through 99 usages of {{globStatus(Path)}} in the following components: hadoop-common, hadoop-distcp, hadoop-rumen, hadoop-streaming, hadoop-yarn-*, hadoop-hdfs (test only). Most {{globStatus}} callers check both {{listFileStatus not null}} and {{listFileStatus.length > 0}}, or expect {{listFileStatus}} to be not {{null}}, except: {code:title=fs.shell.PathData} public static PathData[] expandAsGlob(String pattern, Configuration conf) throws IOException { Path globPath = new Path(pattern); FileSystem fs = globPath.getFileSystem(conf); FileStatus[] stats = fs.globStatus(globPath); PathData[] items = null; if (stats == null) { // remove any quoting in the glob pattern pattern = pattern.replaceAll("\\\\(.)", "$1"); // not a glob & file not found, so add the path with a null stat items = new PathData[]{ new PathData(fs, pattern, null) }; } else { {code} I will include the fix for {{expandAsGlob}} in the next patch. > FileSystem#listStatus should throw IOE upon access error > -------------------------------------------------------- > > Key: HADOOP-7352 > URL: https://issues.apache.org/jira/browse/HADOOP-7352 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Affects Versions: 2.6.0 > Reporter: Matt Foley > Assignee: John Zhuge > Attachments: HADOOP-7352.001.patch > > > In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should > throw FileNotFoundException instead of returning null, when the target > directory did not exist. > However, in LocalFileSystem implementation today, FileSystem::listStatus > still may return null, when the target directory exists but does not grant > read permission. This causes NPE in many callers, for all the reasons cited > in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for > examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org