garyli1019 commented on a change in pull request #1377: [HUDI-663] Fix
HoodieDeltaStreamer offset not handled correctly
URL: https://github.com/apache/incubator-hudi/pull/1377#discussion_r389089300
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java
##########
@@ -180,7 +180,7 @@ public KafkaOffsetGen(TypedProperties props) {
.map(x -> new TopicPartition(x.topic(),
x.partition())).collect(Collectors.toSet());
// Determine the offset ranges to read from
- if (lastCheckpointStr.isPresent()) {
+ if (lastCheckpointStr.isPresent() && !lastCheckpointStr.get().isEmpty())
{
Review comment:
Right. As you mentioned it is still possible that some wrong user behaviors
might lead to an empty checkpoint. From a user perspective, I'd say if there is
an empty checkpoint in the last commit, I will prefer to let the job fail other
than automatically reset the checkpoint. Throw an exception if the checkpoint
is empty would make more sense to me and let the user decide whether they wanna
reset or not. WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
