zhongyujiang commented on pull request #4117: URL: https://github.com/apache/iceberg/pull/4117#issuecomment-1042668763
> @zhongyujiang Flink default only has 1 concurrent checkpoint. could the scenario you described happen in this case? I think it's not relevant to Flink `checkpoint ` but to `notifyCheckpointComplete` working mechanism, quoted from method docs of `notifyCheckpointComplete`: > Notifies the listener that the checkpoint with the given checkpointId completed and was committed. These notifications are "best effort", meaning they can sometimes be skipped. To behave properly, implementers need to follow the "Checkpoint Subsuming Contract". Please see the class-level JavaDocs for details. Please note that checkpoints may generally overlap, so you cannot assume that the notifyCheckpointComplete() call is always for the latest prior checkpoint (or snapshot) that was taken on the function/operator implementing this interface. It might be for a checkpoint that was triggered earlier. Implementing the "Checkpoint Subsuming Contract" (see above) properly handles this situation correctly as well. Please note that throwing exceptions from this method will not cause the completed checkpoint to be revoked. Throwing exceptions will typically cause task/job failure and trigger recovery. `IcebergFilesCommitter` will commit to Iceberg once get notified, and such notifications does not have the same ordering guarantee as Flink checkpoint. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
