nsivabalan commented on code in PR #14085:
URL: https://github.com/apache/hudi/pull/14085#discussion_r2429571656
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java:
##########
@@ -260,19 +261,13 @@ public final <T> ClosableIterator<T>
getEngineRecordIterator(HoodieReaderContext
* @param <T> The type of engine-specific record representation to
return.
* @return An iterator containing the records of interest in specified type.
*/
- public final <T> ClosableIterator<T>
getEngineRecordIterator(HoodieReaderContext<T> readerContext, List<String>
keys, boolean fullKey) {
+ public final <T> ClosableIterator<T>
getEngineRecordIterator(HoodieReaderContext<T> readerContext, List<String>
keys, boolean fullKey) throws IOException {
boolean fullScan = keys.isEmpty();
-
- // Otherwise, we fetch all the records and filter out all the records, but
the
- // ones requested
- ClosableIterator<T> allRecords = getEngineRecordIterator(readerContext);
- if (fullScan) {
- return allRecords;
+ if (!fullScan) {
+ return lookupEngineRecords(keys, fullKey);
+ } else {
+ return ClosableIterator.wrap(Collections.emptyIterator());
}
Review Comment:
In other words, once we call `LogRecordReader.Builder.build()`, based on
whether we are doing FILES partition lookup or any other partition lookup
(keySpec + predicate), we would read all log blocks and fetch the records.
Based on whether keySpec + predicate is set, we will use the right
keyPrefixIterator. And there is no re-use involved here. Reuse is only involved
incase of FILES partition since thats the only one being used in
TimelineServer. Every other partition in MDT does not use the Reuse Record
buffer.
this is different from how we were reading from MDT partitions in 0.14.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
