yihua commented on code in PR #19388:
URL: https://github.com/apache/hudi/pull/19388#discussion_r3668258842
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/InstantComparatorV2.java:
##########
@@ -70,4 +70,14 @@ public Comparator<HoodieInstant>
requestedTimeOrderedComparator() {
public Comparator<HoodieInstant> completionTimeOrderedComparator() {
return COMPLETION_TIME_BASED_COMPARATOR;
}
+
+ @Override
+ public Comparator<HoodieInstant> orderingComparator() {
+ return COMPLETION_TIME_BASED_COMPARATOR;
+ }
+
+ @Override
+ public String getOrderingTime(HoodieInstant instant) {
+ return instant.getCompletionTime();
+ }
Review Comment:
During the upgrade from table version 6 to version 8, the completion time is
backfilled based on the file modification time of the completed commits, and
full compaction is done. Note that the completion time only matters when there
are concurrent writers and when determining MOR file slices. During the
upgrade, there must be no other writers on the same table; given that full
compaction is done, the completion time of completed commits are no longer
useful. So such completion time backfill leveraging the file modification time
is safe. However, in table version 6, the file modification time is not a
reliable way of determining the ordering of instants (not materialized into the
instant file name and metadata), thus table version 6 should only use requested
time for ordering instants, same as Hudi 0.x releases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
