Hi Hang,
You got it right. The problem is exactly at the same line where you pointed [1].
I have used below solution as of now.
```
If(!Files.isReadable(Paths.get(fileStatus.getPath().getPath()))) {
throw new FlinkRuntimeException("Cannot list files under " +
fileStatus.getPath());
}
final FileStatus[] containedFiles = fs.listStatus(fileStatus.getPath());
for (FileStatus containedStatus : containedFiles) {
addSplitsForPath(containedStatus, fs, target);
}
```
Although, if you go inside localf.list(), it checks automatically for the read
permission using Security Manager. This check is getting skipped as Security
Manager is coming as null.
Hence I suspected towards Security Manager.
[1]
https://github.com/apache/flink/blob/9b1375520b6b351df7551d85fcecd920e553cc3a/flink-core/src/main/java/org/apache/flink/core/fs/local/LocalFileSystem.java#L161C32-L161C38<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-722ab8a60d77d5b6&q=1&e=634cbd0d-6962-4ee2-bb8d-7f771a0d428c&u=https%3A%2F%2Fgithub.com%2Fapache%2Fflink%2Fblob%2F9b1375520b6b351df7551d85fcecd920e553cc3a%2Fflink-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fflink%2Fcore%2Ffs%2Flocal%2FLocalFileSystem.java%23L161C32-L161C38>
Regards,
Kirti Dhar
From: Hang Ruan <ruanhang1...@gmail.com>
Sent: Wednesday, March 6, 2024 6:46 PM
To: Kirti Dhar Upadhyay K <kirti.k.dhar.upadh...@ericsson.com>
Cc: User@flink.apache.org
Subject: Re: SecurityManager in Flink
Hi, Kirti.
Could you please provide the stack trace of this NPE? I check the code and I
think maybe the problem lies in LocalFileSystem#listStatus.
The code in line 161[1] may return null, which will let
LocalFileSystem#listStatus return null. Then the `containedFiles` is null and
the NPE occurs.
I think we should add code to handle this situation as follows.
```
final FileStatus[] containedFiles = fs.listStatus(fileStatus.getPath());
if (containedFiles == null) {
throw new FlinkRuntimeException("Cannot list files under " +
fileStatus.getPath());
}
for (FileStatus containedStatus : containedFiles) {
addSplitsForPath(containedStatus, fs, target);
}
```
Best,
Hang
[1]
https://github.com/apache/flink/blob/9b1375520b6b351df7551d85fcecd920e553cc3a/flink-core/src/main/java/org/apache/flink/core/fs/local/LocalFileSystem.java#L161C32-L161C38<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-722ab8a60d77d5b6&q=1&e=634cbd0d-6962-4ee2-bb8d-7f771a0d428c&u=https%3A%2F%2Fgithub.com%2Fapache%2Fflink%2Fblob%2F9b1375520b6b351df7551d85fcecd920e553cc3a%2Fflink-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fflink%2Fcore%2Ffs%2Flocal%2FLocalFileSystem.java%23L161C32-L161C38>
Kirti Dhar Upadhyay K via user
<user@flink.apache.org<mailto:user@flink.apache.org>> 于2024年3月6日周三 18:10写道：
Hi Team,
I am using Flink File Source with Local File System.
I am facing an issue, if source directory does not has read permission, it is
returning the list of files as null instead of throwing permission exception
(refer the highlighted line below), resulting in NPE.
final FileStatus[] containedFiles = fs.listStatus(fileStatus.getPath());
for (FileStatus containedStatus : containedFiles) {
addSplitsForPath(containedStatus, fs, target);
}
Debugging the issue found that, SecurityManager is coming as null while listing
the files, hence skipping the permissions on directory.
What is the way to set SecurityManager in Flink?
Regards,
Kirti Dhar
