nsivabalan commented on code in PR #12718:
URL: https://github.com/apache/hudi/pull/12718#discussion_r1932645713
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/IncrSourceHelper.java:
##########
@@ -195,9 +198,15 @@ public static IncrementalQueryAnalyzer
getIncrementalQueryAnalyzer(
RangeType rangeType;
if (lastCheckpoint.isPresent() &&
!lastCheckpoint.get().getCheckpointKey().isEmpty()) {
+ // User might override checkpoint based on
+ // - instant request time: Then we will treat it as a V1 checkpoint.
+ // - completion time: We will treat it as a normal V2 checkpoint.
+ if (lastCheckpoint.get() instanceof StreamerCheckpointFromCfgCkp) {
+ lastCheckpoint =
Option.of(resolveToV1V2Checkpoint((StreamerCheckpointFromCfgCkp)
lastCheckpoint.get()));
Review Comment:
lets move this resolution (V1 or V2 checkpoint) to translatecheckpoint()
override method in HoodieIncrSource.
So, by the time we reach fetchNextBatch, its going to be either of
StreamerCheckpointV2 or StreamerCheckpointV1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
