nsivabalan commented on a change in pull request #3762:
URL: https://github.com/apache/hudi/pull/3762#discussion_r733063911
##########
File path:
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java
##########
@@ -169,16 +181,98 @@ private void initIfNeeded() {
}
}
+ protected List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>>
getRecordsByKeysFromMetadata(List<String> keys, String partitionName) {
+ Pair<HoodieFileReader, HoodieMetadataMergedLogRecordReader> readers =
openReadersIfNeeded(keys.get(0), partitionName);
+ try {
+ List<Long> timings = new ArrayList<>();
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ HoodieFileReader baseFileReader = readers.getKey();
+ HoodieMetadataMergedLogRecordReader logRecordScanner =
readers.getRight();
+
+ List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>> result =
new ArrayList<>();
+ // local map to assist in merging with base file records
+ Map<String, Option<HoodieRecord<HoodieMetadataPayload>>> logRecords =
new HashMap<>();
+
+ // Retrieve records from log file
+ timer.startTimer();
Review comment:
yeah. this was my thinking.
api in LogRecordScanner to read a single record returns
`Option<HoodieRecord<HoodieMetadataPayload>>`
where as the list of keys returns
`List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>>`.
former one has been there from the beginning and the later one is what we
are adding now.
Unifying them could mean that getRecord for a single key also returns
`List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>>`
but it wasn't intuitive as to why a list is being returned for a single key
look up. and left it as is. But I see your point about managing it in the long
run.
I will probably change the single key method also to return a list. and let
callers handle it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
