n3nash commented on a change in pull request #2611:
URL: https://github.com/apache/hudi/pull/2611#discussion_r587117745
##########
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieHiveUtils.java
##########
@@ -40,6 +44,8 @@
public static final String HOODIE_CONSUME_MODE_PATTERN =
"hoodie.%s.consume.mode";
public static final String HOODIE_START_COMMIT_PATTERN =
"hoodie.%s.consume.start.timestamp";
public static final String HOODIE_MAX_COMMIT_PATTERN =
"hoodie.%s.consume.max.commits";
+ public static final String HOODIE_CONSUME_PENDING_COMMITS =
"hoodie.%s.consume.pending.commits";
+ public static final String HOODIE_CONSUME_VALIDATE_TIMESTAMP =
"hoodie.%s.consume.validate.commit";
Review comment:
s/hoodie.%s.consume.validate.commit/hoodie.%s.consume.commit
We can just treat this config for time-travel. if consume.pending.commits is
enabled, this time-travel will work on inflight, if it's disabled, this
time-travel will work only for completed commit. For now, you can just
implement this for your use-case and later we can implement this for
non-inflight ones. This way we don't leak the specific use-case like
"validation" into the naming.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
