Davis-Zhang-Onehouse commented on code in PR #12687:
URL: https://github.com/apache/hudi/pull/12687#discussion_r1929475728
##########
hudi-common/src/main/java/org/apache/hudi/common/table/checkpoint/CheckpointUtils.java:
##########
@@ -24,19 +24,28 @@
import org.apache.hudi.common.table.HoodieTableVersion;
import org.apache.hudi.common.table.timeline.HoodieInstant;
import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.table.timeline.TimelineUtils;
import org.apache.hudi.common.util.Option;
import org.apache.hudi.common.util.StringUtils;
import org.apache.hudi.exception.HoodieException;
+import java.util.Arrays;
+import java.util.HashSet;
import java.util.Objects;
+import java.util.Set;
import static
org.apache.hudi.common.table.checkpoint.StreamerCheckpointV1.STREAMER_CHECKPOINT_KEY_V1;
import static
org.apache.hudi.common.table.checkpoint.StreamerCheckpointV1.STREAMER_CHECKPOINT_RESET_KEY_V1;
import static
org.apache.hudi.common.table.checkpoint.StreamerCheckpointV2.STREAMER_CHECKPOINT_KEY_V2;
import static
org.apache.hudi.common.table.checkpoint.StreamerCheckpointV2.STREAMER_CHECKPOINT_RESET_KEY_V2;
+import static
org.apache.hudi.common.table.timeline.TimelineUtils.HollowCommitHandling.USE_TRANSITION_TIME;
public class CheckpointUtils {
+ public static final Set<String> DATASOURCES_MUST_USE_CKP_V1 = new
HashSet<>(Arrays.asList(
+ "org.apache.hudi.utilities.sources.S3EventsHoodieIncrSource",
Review Comment:
that's not ideal because:
this general purpose utility method "shouldTargetCheckpointV2" is called
everywhere, where not all the call site have access to class definition of
them. It requires non trivial amount of code refactoring to move code to some
common place where everyone can access.
As compensation, I added unit test validation for
S3EventsHoodieIncrSource.class.getClassName and Gcs. This can also guard
against class moving to other packages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
