xushiyan commented on code in PR #8876:
URL: https://github.com/apache/hudi/pull/8876#discussion_r1226917889
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieCommonConfig.java:
##########
@@ -72,13 +76,23 @@ public class HoodieCommonConfig extends HoodieConfig {
.markAdvanced()
.withDocumentation("Turn on compression for BITCASK disk map used by the
External Spillable Map");
- public static final ConfigProperty<Boolean> READ_BY_STATE_TRANSITION_TIME =
ConfigProperty
- .key("hoodie.datasource.read.by.state.transition.time")
- .defaultValue(false)
+ public static final ConfigProperty<String>
INCREMENTAL_READ_HANDLE_HOLLOW_COMMIT = ConfigProperty
+ .key("hoodie.datasource.read.handle.hollow.commit")
+ .defaultValue(HollowCommitHandling.EXCEPTION.name())
.sinceVersion("0.14.0")
- .withDocumentation("For incremental mode, whether to enable to pulling
commits in range by state transition time(completion time) "
- + "instead of commit time(start time). Please be aware that enabling
this will result in"
- + "`begin.instanttime` and `end.instanttime` using
`stateTransitionTime` instead of the instant's commit time.");
+ .markAdvanced()
+ .withValidValues(enumNames(HollowCommitHandling.class))
+ .withDocumentation("When doing incremental queries, there could be
hollow commits (requested or inflight commits that are not the latest)"
+ + " that are produced by concurrent writers and could lead to
potential data loss. This config allows users to have different ways of
handling this situation."
+ + " The valid values are " +
Arrays.toString(enumNames(HollowCommitHandling.class)) + ":"
+ + " Use `" + HollowCommitHandling.EXCEPTION + "` to throw an
exception when hollow commit is detected. This is helpful when hollow commits"
+ + " are not expected."
+ + " Use `" + HollowCommitHandling.FILTER + "` to stop processing
commits beyond hollow ones. This fits the case where waiting for hollow commits"
Review Comment:
Thinking of changing this to `STOP`
```suggestion
+ " Use `" + HollowCommitHandling.STOP + "` to stop processing
commits beyond hollow ones. This fits the case where waiting for hollow commits"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
