nsivabalan commented on a change in pull request #4334:
URL: https://github.com/apache/hudi/pull/4334#discussion_r773234447
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/S3EventsHoodieIncrSource.java
##########
@@ -83,6 +83,11 @@ public S3EventsHoodieIncrSource(
int numInstantsPerFetch = props.getInteger(NUM_INSTANTS_PER_FETCH,
DEFAULT_NUM_INSTANTS_PER_FETCH);
boolean readLatestOnMissingCkpt = props.getBoolean(
READ_LATEST_INSTANT_ON_MISSING_CKPT,
DEFAULT_READ_LATEST_INSTANT_ON_MISSING_CKPT);
+ IncrSourceHelper.MissingCheckpointStrategy missingCheckpointStrategy =
(props.containsKey(HoodieIncrSource.Config.MISSING_CHECKPOINT_STRATEGY))
+ ?
IncrSourceHelper.MissingCheckpointStrategy.valueOf(props.getString(HoodieIncrSource.Config.MISSING_CHECKPOINT_STRATEGY))
: null;
Review comment:
nope. default value for READ_LATEST_INSTANT_ON_MISSING_CKPT is false.
So, we can't set default strategy for the new config.
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/IncrSourceHelper.java
##########
@@ -32,6 +32,16 @@
public class IncrSourceHelper {
+ /**
+ * Kafka reset offset strategies.
+ */
+ public enum MissingCheckpointStrategy {
+ // read from latest commit in hoodie source table
+ READ_LATEST,
Review comment:
yeah, this sounds better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
