[
https://issues.apache.org/jira/browse/HADOOP-7352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472754#comment-15472754
]
John Zhuge commented on HADOOP-7352:
------------------------------------
Move review comments from HADOOP-13191 to here.
Steve commented:
bq. Be aware that the FS shell expects globStatus to return null in certain
conditions: someone needs to look at all uses of this call and make sure that
it isn't being used
John commented:
Went through 99 usages of {{globStatus(Path)}} in the following components:
hadoop-common, hadoop-distcp, hadoop-rumen, hadoop-streaming, hadoop-yarn-*,
hadoop-hdfs (test only).
Most {{globStatus}} callers check both {{listFileStatus not null}} and
{{listFileStatus.length > 0}}, or expect {{listFileStatus}} to be not {{null}},
except:
{code:title=fs.shell.PathData}
public static PathData[] expandAsGlob(String pattern, Configuration conf)
throws IOException {
Path globPath = new Path(pattern);
FileSystem fs = globPath.getFileSystem(conf);
FileStatus[] stats = fs.globStatus(globPath);
PathData[] items = null;
if (stats == null) {
// remove any quoting in the glob pattern
pattern = pattern.replaceAll("\\\\(.)", "$1");
// not a glob & file not found, so add the path with a null stat
items = new PathData[]{ new PathData(fs, pattern, null) };
} else {
{code}
I will include the fix for {{expandAsGlob}} in the next patch.
> FileSystem#listStatus should throw IOE upon access error
> --------------------------------------------------------
>
> Key: HADOOP-7352
> URL: https://issues.apache.org/jira/browse/HADOOP-7352
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.6.0
> Reporter: Matt Foley
> Assignee: John Zhuge
> Attachments: HADOOP-7352.001.patch
>
>
> In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should
> throw FileNotFoundException instead of returning null, when the target
> directory did not exist.
> However, in LocalFileSystem implementation today, FileSystem::listStatus
> still may return null, when the target directory exists but does not grant
> read permission. This causes NPE in many callers, for all the reasons cited
> in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for
> examples.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
