danny0405 commented on code in PR #10945:
URL: https://github.com/apache/hudi/pull/10945#discussion_r1548805924
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1375,33 +1375,15 @@ public static Set<String>
getValidInstantTimestamps(HoodieTableMetaClient dataMe
/**
* Checks if the Instant is a delta commit and has a valid suffix for
operations on MDT.
*
+ * @param datasetPendingInstants The dataset pending instants
* @param instant {@code HoodieInstant} to check.
* @return {@code true} if the instant is valid.
*/
- public static boolean isValidInstant(HoodieInstant instant) {
- // Should be a deltacommit
- if (!instant.getAction().equals(HoodieTimeline.DELTA_COMMIT_ACTION)) {
- return false;
- }
-
- // Check correct length. The timestamp should have a suffix over the
timeline's timestamp format.
- final String instantTime = instant.getTimestamp();
- if (!(instantTime.length() == MILLIS_INSTANT_ID_LENGTH +
OperationSuffix.METADATA_INDEXER.getSuffix().length())) {
- return false;
- }
-
- // Is this a fixed operations suffix
- final String suffix = instantTime.substring(instantTime.length() - 3);
- if (OperationSuffix.isValidSuffix(suffix)) {
- return true;
- }
-
- // Is this a index init suffix?
- if (suffix.compareTo(String.format("%03d",
PARTITION_INITIALIZATION_TIME_SUFFIX)) >= 0) {
- return true;
- }
Review Comment:
It is not necessary any more, after the change, any delta_commit that with
auto-generated timestamp by the MDT should be valid, and we only need to
filter out those pending instant (completed in MDT but fails in DT).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
