[
https://issues.apache.org/jira/browse/HADOOP-7327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043247#comment-13043247
]
Matt Foley commented on HADOOP-7327:
------------------------------------
We have two instances of this bug actually causing NPEs in the real world (or
at least the Test world).
The first is in TestTrash.testTrashEmptier(), which runs on the
LocalFileSystem, and launches a Trash$Emptier thread in background. Every time
that thread cycles, it attempts to empty the local .Trash directory in EVERY
"home directory" known. On my Mac, there are four user accounts, each with its
own "home" directory, one of which, /Users/test, has a .Trash directory with
access permissions set so it is not readable by me. When it tries to list the
.Trash subdirectories with FileSystem.listStatus(), it logs the following:
11/05/23 14:34:54 WARN fs.Trash: RuntimeException during Trash.Emptier.run()
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1114)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1137)
at
org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
at org.apache.hadoop.fs.Trash.expunge(Trash.java:215)
at org.apache.hadoop.fs.Trash$Emptier.run(Trash.java:313)
at java.lang.Thread.run(Thread.java:680)
The second example is from Daryn, and was partly described in HADOOP-7345.
When he tries to use "bin/hadoop fs -ls aaa" on any directory "aaa" for which
he does not have access privs, it logs:
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1115)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1150)
at
org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
at
org.apache.hadoop.fs.shell.PathData.getDirectoryContents(PathData.java:163)
It is seen that both of these cases go through the
FileSystem.listStatus(ArrayList<FileStatus>, Path, PathFilter) form of the
call, and it is here that the attached patch is targeted.
> FileSystem.listStatus() throws NullPointerException instead of IOException
> upon access permission failure
> ---------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7327
> URL: https://issues.apache.org/jira/browse/HADOOP-7327
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Attachments: hadoop-7327-1.patch
>
>
> Many processes that call listStatus() expect to handle IOException, but
> instead are getting runtime error NullPointerException, if the directory
> being scanned is visible but no-access to the running user id. For example,
> if directory foo is drwxr-xr-x, and subdirectory foo/bar is drwx------, then
> trying to do listStatus(Path(foo/bar)) will cause a NullPointerException.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
