[
https://issues.apache.org/jira/browse/HADOOP-13191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15471785#comment-15471785
]
John Zhuge commented on HADOOP-13191:
-------------------------------------
Went through 99 usages of {{globStatus(Path)}} in the following components:
hadoop-common, hadoop-distcp, hadoop-rumen, hadoop-streaming, hadoop-yarn-*,
hadoop-hdfs (test only).
Most {{globStatus}} callers check both {{listFileStatus not null}} and
{{listFileStatus.length > 0}}, or expect {{listFileStatus}} to be not {{null}},
except:
{code:title=fs.shell.PathData}
public static PathData[] expandAsGlob(String pattern, Configuration conf)
throws IOException {
Path globPath = new Path(pattern);
FileSystem fs = globPath.getFileSystem(conf);
FileStatus[] stats = fs.globStatus(globPath);
PathData[] items = null;
if (stats == null) {
// remove any quoting in the glob pattern
pattern = pattern.replaceAll("\\\\(.)", "$1");
// not a glob & file not found, so add the path with a null stat
items = new PathData[]{ new PathData(fs, pattern, null) };
} else {
{code}
> FileSystem#listStatus should not return null
> --------------------------------------------
>
> Key: HADOOP-13191
> URL: https://issues.apache.org/jira/browse/HADOOP-13191
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.6.0
> Reporter: John Zhuge
> Assignee: John Zhuge
> Priority: Minor
> Attachments: HADOOP-13191.001.patch, HADOOP-13191.002.patch,
> HADOOP-13191.003.patch, HADOOP-13191.004.patch
>
>
> This came out of discussion in HADOOP-12718. The {{FileSystem#listStatus}}
> contract does not indicate {{null}} is a valid return and some callers do not
> test {{null}} before use:
> AbstractContractGetFileStatusTest#testListStatusEmptyDirectory:
> {code}
> assertEquals("ls on an empty directory not of length 0", 0,
> fs.listStatus(subfolder).length);
> {code}
> ChecksumFileSystem#copyToLocalFile:
> {code}
> FileStatus[] srcs = listStatus(src);
> for (FileStatus srcFile : srcs) {
> {code}
> SimpleCopyLIsting#getFileStatus:
> {code}
> FileStatus[] fileStatuses = fileSystem.listStatus(path);
> if (excludeList != null && excludeList.size() > 0) {
> ArrayList<FileStatus> fileStatusList = new ArrayList<>();
> for(FileStatus status : fileStatuses) {
> {code}
> IMHO, there is no good reason for {{listStatus}} to return {{null}}. It
> should throw IOExceptions upon errors or return empty list.
> To enforce the contract that null is an invalid return, update javadoc and
> leverage @Nullable/@NotNull/@Nonnull annotations.
> So far, I am only aware of the following functions that can return null:
> * RawLocalFileSystem#listStatus
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
