lxy-9602 commented on PR #564: URL: https://github.com/apache/paimon-cpp/pull/564#issuecomment-6944276366
Non-blocking: the four-reader window is scoped to each split. `CreateReader(splits)` still eagerly creates a reader for every split, so with S splits containing at least four files each, it can retain up to 4 × S file readers before returning the first batch. Early termination also pays the initial preparation cost for splits that are never consumed. Could we consider lazy initialization at the split level as a follow-up? If that requires substantial changes, leaving a TODO near the multi-split creation loop would be sufficient for this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
