garyli1019 commented on issue #1362: HUDI-644 Enable user to get checkpoint from previous commits in DeltaStreamer URL: https://github.com/apache/incubator-hudi/pull/1362#issuecomment-595908998 > Let me catch up on this discussion and circle back.. :) > > Just one high level question (apologies if its already answered above). > > why can't we use the checkpoint reset flag, if one-time manual restarts are needed for deltastreamer? is it because its hard to compute that? Right. I need a robust way to generate the checkpoint from kafka-connect-hdfs managed files and kafka-connect itself sometimes having an issue to retrieve checkpoint when the Kafka partition number was large. The mechanism is to scan every single file and get the latest checkpoint of each Kafka partition.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
