SteNicholas commented on code in PR #576:
URL: https://github.com/apache/paimon-cpp/pull/576#discussion_r4620845131


##########
src/paimon/common/reader/concat_batch_reader.h:
##########
@@ -47,14 +48,17 @@ class ConcatBatchReader : public BatchReader {
     /// Called at most once per index, possibly concurrently. A null reader 
means a skipped file.
     using ReaderFactory = 
std::function<Result<std::unique_ptr<BatchReader>>(size_t)>;
 
-    static constexpr size_t kPreparationWindow = kReaderBuildMaxParallelNum;
+    static constexpr size_t kDefaultPreparationWindow = 4;
 
     /// Builds the initial window concurrently and refills it as readers reach 
EOF. Later
     /// construction failures are terminal read errors when the failed reader 
is reached.
+    /// The positive preparation window bounds retained readers, including 
in-flight builds,
+    /// independently of the build executor's parallelism.
     static Result<std::unique_ptr<ConcatBatchReader>> CreateLazy(
         size_t reader_count, ReaderFactory factory,
         const std::shared_ptr<arrow::MemoryPool>& arrow_pool,
-        const std::shared_ptr<Executor>& build_executor = 
GetReaderBuildExecutor());
+        const std::shared_ptr<Executor>& build_executor = 
GetReaderBuildExecutor(),
+        int32_t preparation_window = kDefaultPreparationWindow);

Review Comment:
   Removed the default executor argument from `CreateLazy` and updated its 
callers to pass the executor explicitly. `preparation_window` now comes before 
`arrow_pool` and `build_executor`, keeping the pool and executor at the end. 
[Updated 
signature](https://github.com/apache/paimon-cpp/blob/ebe1e01b34628cff5c3893719e4b0af889f147d9/src/paimon/common/reader/concat_batch_reader.h#L56-L59)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to