codope commented on code in PR #12105:
URL: https://github.com/apache/hudi/pull/12105#discussion_r1805202845
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -931,17 +939,22 @@ private Map<String,
List<HoodieRecord<HoodieMetadataPayload>>> lookupSecondaryKe
List<String> sortedSecondaryKeys = new ArrayList<>(secondaryKeys);
Collections.sort(sortedSecondaryKeys);
secondaryKeySet.addAll(sortedSecondaryKeys);
+ Set<String> deletedRecordKeysFromLogs = new HashSet<>();
logRecordScanner.getRecords().forEach(record -> {
HoodieMetadataPayload payload = record.getData();
- String recordKey = payload.getRecordKeyFromSecondaryIndex();
- if (secondaryKeySet.contains(recordKey)) {
- String secondaryKey = payload.getRecordKeyFromSecondaryIndex();
- logRecordsMap.computeIfAbsent(secondaryKey, k -> new
HashMap<>()).put(recordKey, record);
+ if (!payload.isDeleted()) {
+ String recordKey = payload.getRecordKeyFromSecondaryIndex();
+ if (secondaryKeySet.contains(recordKey)) {
+ String secondaryKey = payload.getRecordKeyFromSecondaryIndex();
Review Comment:
I was just discussing this fix with @lokeshj1703 today. He has a patch too -
#12119 (though it does not filter out deleted records). If the sec index as
changed in this commit are passing, then we can close #12119 in favor of your
patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
