yihua commented on code in PR #12986:
URL: https://github.com/apache/hudi/pull/12986#discussion_r1999848876
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -824,7 +825,8 @@ public Map<String, Set<String>>
getSecondaryIndexRecords(List<String> keys, Stri
return Collections.emptyMap();
}
- return getRecordsByKeyPrefixes(keys, partitionName, false).map(
+ return getAllRecordsByKeys(keys,
partitionName).values().stream().flatMap(Collection::stream)
+ .map(
Review Comment:
Since the file group mapping changes from table version 8 to 9, we should
differentiate the reading and look up logic based on the table version, to
avoid wrong results (note that the secondary index layout changes on storage).
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1339,7 +1340,14 @@ private static List<Tuple3<String, String, Boolean>>
fetchPartitionFileInfoTripl
* @param recordKey record key for which the file group index is looked up
for.
* @return An integer hash of the given string
*/
- public static int mapRecordKeyToFileGroupIndex(String recordKey, int
numFileGroups) {
+ public static int mapRecordKeyToFileGroupIndex(String recordKey, int
numFileGroups, String partitionName) {
+ if (MetadataPartitionType.SECONDARY_INDEX.isPartitionType(partitionName)
&& recordKey.contains(SECONDARY_INDEX_RECORD_KEY_SEPARATOR)) {
+ return
mapRecordKeyToFileGroupIndex(SecondaryIndexKeyUtils.getSecondaryKeyFromSecondaryIndexKey(recordKey),
numFileGroups);
+ }
+ return mapRecordKeyToFileGroupIndex(recordKey, numFileGroups);
Review Comment:
We should guard this with write table version, i.e., the new logic should
only take effect for the new table version 9.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
