simonsssu edited a comment on pull request #1515: URL: https://github.com/apache/iceberg/pull/1515#issuecomment-707445254
@rdblue Hi Ryan, It's a good suggestion, maybe I can open another PR to add the configuration check for Iceberg Sink without checkpoint. I think for most Flink cases, if we don't enable checkpoint, it was at-most-once guarantee because it will lost the state generated during job failover, unless we manually read data from beginning. So when using Flink Iceberg Sink in most production cases I think it's better to enable checkpoint. Currently we recover the state with a list of DataFiles, which are persisted by Committer. Your ideas is to make IcebergWriter to recover the state by finding the last successful commit , am I right ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
