tonymtu opened a new pull request, #11247: URL: https://github.com/apache/paimon/pull/11247
### Purpose Directory discovery currently uses `FileIO.listStatus`, which materializes all entries. The existing `listFilesIterative` returns files only. Add `FileIO.listStatusIterative(Path)` to iterate over the immediate files and subdirectories, with a default implementation backed by `listStatus`. The Jindo implementation pages through the existing SDK iterator. Add an optional `FileStatus.getStorageClass()` for callers choosing which objects to process. Both Jindo directory listing methods expose the class already present in the listing. A directory, a single-file lookup, or an unreported class returns `null`. Reading the attribute performs no I/O. ### Tests - Built the Jindo plugin and its dependencies. All 115 selected FileIO and Jindo tests passed, including wrapper delegation, page errors and the single-file case. - Tested on OSS with Jindo 6.9.1, 100 keys per page, 205 immediate files across five storage classes, and one subdirectory. All three methods returned the same files over six traversals each. Both directory listing methods returned the subdirectory and the correct class for every file. Also checked a single Archive object and an empty directory. OSS access logs gave the following request counts per traversal. All metadata HEAD requests targeted the directory. | Method | Data page LIST | Directory probe LIST | Directory HEAD | Total | | --- | ---: | ---: | ---: | ---: | | `listFilesIterative(path, false)` | 3 | 3 | 6 | 12 | | `listStatusIterative(path)` | 3 | 3 | 6 | 12 | | `listStatus(path)` | 3 | 1 | 2 | 6 | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
