ChenSammi commented on code in PR #3711:
URL: https://github.com/apache/ozone/pull/3711#discussion_r971732530
##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneFileSystem.java:
##########
@@ -966,6 +966,135 @@ public RemoteIterator<LocatedFileStatus>
listLocatedStatus(Path f)
return super.listLocatedStatus(f);
}
+ @Override
+ public RemoteIterator<FileStatus> listStatusIterator(Path f)
+ throws IOException {
+ return new OzoneFileStatusIterator<>(f);
+ }
+
+ /**
+ * A private class implementation for iterating list of file status.
+ *
+ * @param <T> the type of the file status.
+ */
+ private final class OzoneFileStatusIterator<T extends FileStatus>
+ implements RemoteIterator<T> {
+ private List<FileStatus> thisListing;
+ private int i;
+ private Path p;
+ private T curStat = null;
+ private String startPath = "";
+
+ /**
+ * Constructor to initialize OzoneFileStatusIterator.
+ * Get the first batch of entry for iteration.
+ *
+ * @param p path to file/directory.
+ * @throws IOException
+ */
+ private OzoneFileStatusIterator(Path p) throws IOException {
+ this.p = p;
+ // fetch the first batch of entries in the directory
+ thisListing = listFileStatus(p, startPath);
+ if (thisListing != null && !thisListing.isEmpty()) {
+ startPath = pathToKey(
+ thisListing.get(thisListing.size() - 1).getPath());
+ LOG.info("Got {} file status, next start path {}",
Review Comment:
Can you change log level to debug or trace?
##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneFileSystem.java:
##########
@@ -966,6 +966,135 @@ public RemoteIterator<LocatedFileStatus>
listLocatedStatus(Path f)
return super.listLocatedStatus(f);
}
+ @Override
+ public RemoteIterator<FileStatus> listStatusIterator(Path f)
+ throws IOException {
+ return new OzoneFileStatusIterator<>(f);
+ }
+
+ /**
+ * A private class implementation for iterating list of file status.
+ *
+ * @param <T> the type of the file status.
+ */
+ private final class OzoneFileStatusIterator<T extends FileStatus>
+ implements RemoteIterator<T> {
+ private List<FileStatus> thisListing;
+ private int i;
+ private Path p;
+ private T curStat = null;
+ private String startPath = "";
+
+ /**
+ * Constructor to initialize OzoneFileStatusIterator.
+ * Get the first batch of entry for iteration.
+ *
+ * @param p path to file/directory.
+ * @throws IOException
+ */
+ private OzoneFileStatusIterator(Path p) throws IOException {
+ this.p = p;
+ // fetch the first batch of entries in the directory
+ thisListing = listFileStatus(p, startPath);
+ if (thisListing != null && !thisListing.isEmpty()) {
+ startPath = pathToKey(
+ thisListing.get(thisListing.size() - 1).getPath());
+ LOG.info("Got {} file status, next start path {}",
+ thisListing.size(), startPath);
+ }
+ statistics.incrementReadOps(1);
Review Comment:
Don't need to update the statistics since it's already done in
listFileStatus call. Please double check other places too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
