westonpace commented on code in PR #13796:
URL: https://github.com/apache/arrow/pull/13796#discussion_r948850806
##########
cpp/src/arrow/filesystem/localfs.h:
##########
@@ -34,10 +34,24 @@ namespace fs {
/// Options for the LocalFileSystem implementation.
struct ARROW_EXPORT LocalFileSystemOptions {
+ static constexpr int32_t kDefaultDirectoryReadahead = 16;
+ static constexpr int32_t kDefaultFileInfoBatchSize = 1000;
+
/// Whether OpenInputStream and OpenInputFile return a mmap'ed file,
/// or a regular one.
bool use_mmap = false;
+ /// Options related to `GetFileInfoGenerator` interface.
+
+ /// How many directories should be processed in parallel
+ /// by the `GetFileInfoGenerator` impl.
+ int32_t directory_readahead = kDefaultDirectoryReadahead;
+ /// Specifies how much entries shall be aggregated into
+ /// a single FileInfoVector chunk by the `GetFileInfoGenerator` impl, which
+ /// is the result of `stat`:ing individual dirents, obtained by the call to
+ /// `internal::ListDir`.
Review Comment:
This is going to be a rather heavy comment for your average Arrow user. Can
we perhaps just give a brief description of the reasons you might want to
change this parameter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
