Reo-LEI edited a comment on issue #3102: URL: https://github.com/apache/iceberg/issues/3102#issuecomment-917645574
Actually, in our production environment, we use flink upsert an iceberg table and commit every 5mins. But flink job will frequently fail by checkpoint timeout(15min).  I found `IcebergFilesCommitter` will take too many time to travel all snapshot history. So I specify the data file validation start from `lastCommittedSnapshotId`(as #3103). Now the commit time has been reduced from a few minutes to a few seconds.  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
