nsivabalan commented on code in PR #14085:
URL: https://github.com/apache/hudi/pull/14085#discussion_r2429571656
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java:
##########
@@ -260,19 +261,13 @@ public final <T> ClosableIterator<T>
getEngineRecordIterator(HoodieReaderContext
* @param <T> The type of engine-specific record representation to
return.
* @return An iterator containing the records of interest in specified type.
*/
- public final <T> ClosableIterator<T>
getEngineRecordIterator(HoodieReaderContext<T> readerContext, List<String>
keys, boolean fullKey) {
+ public final <T> ClosableIterator<T>
getEngineRecordIterator(HoodieReaderContext<T> readerContext, List<String>
keys, boolean fullKey) throws IOException {
boolean fullScan = keys.isEmpty();
-
- // Otherwise, we fetch all the records and filter out all the records, but
the
- // ones requested
- ClosableIterator<T> allRecords = getEngineRecordIterator(readerContext);
- if (fullScan) {
- return allRecords;
+ if (!fullScan) {
+ return lookupEngineRecords(keys, fullKey);
+ } else {
+ return ClosableIterator.wrap(Collections.emptyIterator());
}
Review Comment:
In other words, once we call `LogRecordReader.Builder.build()`, based on
whether we are doing FILES partition lookup or any other partition lookup
(keySpec + predicate), we would read all log blocks and fetch the records. And
there is no re-use involved here. Reuse is only involved incase of FILES
partition since thats the only one being used in TimelineServer. Every other
partition in MDT does not use the Reuse Record buffer.
this is different from how we were reading from MDT partitions in 0.14.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
