codope commented on code in PR #11256:
URL: https://github.com/apache/hudi/pull/11256#discussion_r1607940757
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java:
##########
@@ -288,12 +288,9 @@ public static <R> HoodieData<HoodieRecord<R>>
mergeForPartitionUpdatesIfNeeded(
// the tagging partitions and locations
// NOTE: The incoming records may only differ in record position, however,
for the purpose of
// merging in case of partition updates, it is safe to ignore the
record positions.
- HoodieData<HoodieRecordGlobalLocation> globalLocations =
incomingRecordsAndLocations
+ HoodieData<Pair<String, String>> globalLocations =
incomingRecordsAndLocations
.filter(p -> p.getRight().isPresent())
- .map(p -> new HoodieRecordGlobalLocation(
- p.getRight().get().getPartitionPath(),
- p.getRight().get().getInstantTime(),
- p.getRight().get().getFileId()))
+ .map(p -> Pair.of(p.getRight().get().getPartitionPath(),
p.getRight().get().getFileId()))
Review Comment:
Wondering if we can simply remove `instantTime` from the `equals` and
`hashcode` of `HoodieRecordGlobalLocation`. Why would we need `instantTime` for
global location? Clustering will anyway map to a new fileId. Compaction will
create a base file with same file id but different instant time. However, for
the location all we care about is partition path and filegroup id right.
Instant time is only used for filesystem view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
