smaheshwar-pltr opened a new issue, #17990: URL: https://github.com/apache/iceberg/issues/17990
### Apache Iceberg version 1.11.0 (latest release) ### Query engine None ### Please describe the bug 🐞 Concurrent commits through one shared, encrypted `Table` can commit a snapshot without the wrapped key required to read its manifest list. The commit succeeds, but the snapshot's key is lost, leaving the table unreadable to users and seemingly lost. See https://github.com/apache/iceberg/pull/17982 for a reproduction, the user-facing error will be something like: ``` java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "org.apache.iceberg.encryption.EncryptedKey.encryptedKeyMetadata()" because "manifestListKey" is null Caused by: java.lang.NullPointerException at org.apache.iceberg.encryption.EncryptionUtil.decryptManifestListKeyMetadata(...) at org.apache.iceberg.BaseSnapshot.allManifests(...) at org.apache.iceberg.FastAppend.apply(...) at org.apache.iceberg.SnapshotProducer.commit(...) ``` This situation surfaces in https://github.com/apache/iceberg/issues/17989, but also more explicitly in https://github.com/apache/iceberg/blob/main/hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveTableConcurrency.java. The root cause is that https://github.com/apache/iceberg/blob/main/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java has race conditions which, for encryption, can lead to key loss. The issue here is that a writer can write a snapshot, mutating its encryption manager, when a concurrent refresh from another writer "installs" a different manager before the first writer's commit, losing the first writer's snapshot keys. Concretely, with two writers: 1. Writer A refreshes the table, constructs its manifest-list writer with encryption manager A, and blocks while that manager unwraps its key-encryption key before publishing the new manifest-list key. 2. Writer B refreshes the same table operations, "installs" encryption manager B, constructs its writer, and blocks at the same point. 3. Writer A resumes and publishes the new manifest-list key to manager A. 4. Writer A's Hive commit copies encryption keys from *manager B*. 5. Writer A's snapshot commits with a manifest-list key ID, but its key is absent from table metadata. 6. Writer B resumes. Its first commit is stale, and its retry fails in `FastAppend.apply` while reading writer A's manifest list because `manifestListKey` is null. 7. After reloading the table from the catalog, writer A's snapshot still lacks its key and planning it therefore fails with the same exception. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
