simonsssu commented on pull request #1515: URL: https://github.com/apache/iceberg/pull/1515#issuecomment-708924017
@kbendick I think the problem now is Flink don't have the API to tell a batch or stream mode, so in current implementation, it's hard to determine a job without checkpoint is valid or not. (Although you can set a config with JAVA API, but it's hard to do this when running IcebergTableSink by SQL. Although we can use a processing time to trigger a commit when disable checkpoint, it will also make a batch job split into several small transaction commit. I think the solution would be: 1. For stream job, we can register a timer for jobs which disable checkpoint. 2. For batch job, we do nothing. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
