lucasfang commented on code in PR #342:
URL: https://github.com/apache/paimon-cpp/pull/342#discussion_r4032759967
##########
include/paimon/reader/prefetch_file_batch_reader.h:
##########
@@ -135,6 +136,20 @@ class PAIMON_EXPORT PrefetchFileBatchReader : public
FileBatchReader {
virtual Result<std::vector<std::pair<uint64_t, uint64_t>>>
PreBufferRange() {
return std::vector<std::pair<uint64_t, uint64_t>>{};
}
+
+ /// Callback a reader reports byte ranges through when they only become
known after reading has
+ /// started, so the prefetch layer can register them with its shared
read-ahead cache.
+ ///
+ /// PreBufferRange() covers what is known up front; a reader whose ranges
depend on data it
+ /// has already read - the late-materialization payload pass only knows
which pages hold the
+ /// matched rows once the probe pass has run - reports them through this
callback instead.
+ using PreBufferRangeCallback =
+ std::function<void(std::vector<std::pair<uint64_t, uint64_t>>&&)>;
+
+ /// Installs the callback above, or clears it when `callback` is empty. By
default a reader has
+ /// no late byte ranges to report and ignores the callback.
+ /// @param callback The callback to report late byte ranges through.
+ virtual void SetPreBufferRangeCallback(PreBufferRangeCallback callback) {}
Review Comment:
I will refactor the decoupling of the read-ahead cache and the prefetch
reader in subsequent steps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
