FangYongs commented on issue #1635: URL: https://github.com/apache/incubator-paimon/issues/1635#issuecomment-1668970661
@JingsongLi This is used for data incremental correction for streaming processing. For example, there's a Flink streaming etl job reads data from kafka and writes results to Paimon and we will regularly trigger savepoints in Flink to ensure job recovery in case of disaster. We sometimes need to redo data for kafka from specified timestamp, and resume Flink streaming etl job from the specified offset. At present, we cannot incrementally recover and can only discard the results of previous computation. But with Flink savepoint and tag in Paimon, we can rollback data in Paimon to given tag and resume Flink job from given savepoint to incremental streaming process without lose any data. The detail of incremental recover is described in `b) Increment Recover` of [PIP-5](https://cwiki.apache.org/confluence/display/PAIMON/PIP-5%3A+Paimon+Table+And+Data+Lineage+For+Flink) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
