Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3033#discussion_r226293537
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetFile.java
---
@@ -308,32 +321,45 @@ public boolean accept(final File file) {
};
}
- private Set<File> performListing(final File directory, final
FileFilter filter, final boolean recurseSubdirectories) {
- Path p = directory.toPath();
- if (!Files.isWritable(p) || !Files.isReadable(p)) {
- throw new IllegalStateException("Directory '" + directory + "'
does not have sufficient permissions (i.e., not writable and readable)");
- }
- final Set<File> queue = new HashSet<>();
- if (!directory.exists()) {
- return queue;
- }
-
- final File[] children = directory.listFiles();
- if (children == null) {
- return queue;
- }
-
- for (final File child : children) {
- if (child.isDirectory()) {
+ private Set<File> performListing(final File directory, final
FileFilter filter, final boolean recurseSubdirectories, final int batchSize) {
+ try {
+ if (directoryStream == null || !this.fileIterator.hasNext()) {
+ final Path p = directory.toPath();
+ if (!Files.isReadable(p) || !Files.isWritable(p)) {
+ throw new IllegalStateException("Directory '" +
directory + "' does not have sufficient permissions (i.e., not writable and
readable)");
+ }
+
+ if (!directory.exists()) {
+ return Collections.emptySet();
+ }
+
+ Stream<Path> listStream;
if (recurseSubdirectories) {
- queue.addAll(performListing(child, filter,
recurseSubdirectories));
+ listStream = Files.walk(p);
--- End diff --
@markap14 hadn't we switched to using the streaming api previously and
backed this out? If so what was the problem? Or maybe that was ListFile?
---
