HeartSaVioR edited a comment on issue #1286: URL: https://github.com/apache/iceberg/issues/1286#issuecomment-668358945
> Or just write to the data file without committing the transaction and after some another interval to commit the transaction. If I understand correctly, this is already the thing we do while rewriting data files (at least Spark action is, but the custom implementation would be similar). The latter is a thing Iceberg already cares about during retrying commit - you can configure max number of retry and interval on retrial to adjust the time to commit. The problem occurs when the commit can't be done between micro-batch interval - in this case it always lose the race. Increasing the max number of retry or interval on retrial wouldn't help. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
