bube opened a new issue, #8470: URL: https://github.com/apache/paimon/issues/8470
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master @ `9ea812059` (also affects 1.x releases with the Iceberg REST committer) ### Compute Engine Flink ### Minimal reproduce step 1. Two Paimon sinks writing to two different tables in the same database, with Iceberg REST catalog sync enabled (`'metadata.iceberg.storage' = 'rest-catalog'`), pointing at the same REST catalog. 2. The Iceberg namespace (database) does not exist beforehand. 3. When the two sinks' first commits happen concurrently, both find the namespace missing and both call `createNamespace()` — one gets raced and fails. Root cause — `IcebergRestMetadataCommitter.commitMetadataImpl()` uses a check-then-act pattern across two separate REST round-trips: ```java // create database if not exist if (!databaseExists()) { createDatabase(); } ``` The loser of the race gets an error creating the already-created namespace: ``` java.lang.RuntimeException: org.apache.iceberg.exceptions.ServiceFailureException: Server error: UncheckedSQLException: Failed to execute: INSERT INTO iceberg_namespace_properties (catalog_name, namespace, property_key, property_value) VALUES (?,?,?,?) at org.apache.paimon.iceberg.IcebergRestMetadataCommitter.commitMetadata(IcebergRestMetadataCommitter.java:127) at org.apache.paimon.iceberg.IcebergCommitCallback.createMetadataWithoutBase(IcebergCommitCallback.java:394) at org.apache.paimon.iceberg.IcebergCommitCallback.createMetadata(IcebergCommitCallback.java:288) at org.apache.paimon.iceberg.IcebergCommitCallback.call(IcebergCommitCallback.java:224) at org.apache.paimon.operation.FileStoreCommitImpl.lambda$tryCommitOnce$11(FileStoreCommitImpl.java:1048) at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:1048) at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:703) at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:327) ``` ### What doesn't meet your expectations? Concurrent commits to different tables in the same (not-yet-created) database should not fail. The raced sink fails its commit and Flink restarts it. It recovers fine after the restart (the namespace exists by then), but this produces spurious failures and error-log/alert noise on the first commit of a fresh database. No data-correctness impact. ### Anything else? The exact exception depends on the REST server's spec compliance: - Per the Iceberg REST spec, a duplicate namespace returns **409 → `AlreadyExistsException`** (`org.apache.iceberg.rest.RESTServerExtension` correctly returns this). - The `apache/iceberg-rest-fixture` docker image incorrectly returns **500 → `ServiceFailureException`** (the stack trace above). Suggested fix: make namespace creation idempotent by catching `AlreadyExistsException` from `createNamespace()` and proceeding, since the required namespace now exists. (A spec-violating 500 from a non-compliant server is a separate, server-side problem.) ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
