GrigorievNick opened a new issue #1131: URL: https://github.com/apache/iceberg/issues/1131
Hi ICBQm community please help me to understand how icbq grant commits consistency on s3 for concurrent writes. From [icbq documentation ](https://iceberg.apache.org/reliability/#concurrent-write-operations) ``` Concurrent write operations Iceberg supports multiple concurrent writes using optimistic concurrency. Each writer assumes that no other writers are operating and writes out new table metadata for an operation. Then, the writer attempts to commit by atomically swapping the new table metadata file for the existing metadata file. If the atomic swap fails because another writer has committed, the failed writer retries by writing a new metadata tree based on the new current table state. ``` But Lest image situation: User one writes data to the table and then commits a new snapshot. But commit where? To **file**(Valid snapshots in an Iceberg table are stored the table metadata file) **on s3.** Because of eventually consistency of s3, the Second User writes new data and going to commit. Before commit, the user checks iceberg table metadata file for snapshots to do an optimistic lock, but S3 is eventually consistent. So the user doesn't see a new Iceberg table metadata file created by the user one and just overwrite this file on s3. This situation leads to the loss of all changes done by the User one. Am I right? or I miss something? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
