vigneshio opened a new issue, #4658: URL: https://github.com/apache/polaris/issues/4658
When Polaris receives a CREATE or UPDATE notification for an external table, the operation can fail if another process modifies the same table at the same time. Currently: * UPDATE fails with a concurrent modification error. * CREATE fails if the table was created by another process before the request completes. * In both cases, the notification fails instead of retrying. Simply retrying the same write is not enough because the table information is read only once, making it stale after a concurrent update. A proper fix should: 1. Re-read the latest table entity on every retry. 2. Apply the new metadata location and timestamp again. 3. Retry the update with a limited number of attempts and backoff. To keep things simple, the initial change could focus only on retrying UPDATE failures caused by concurrent modifications, since that is the most common scenario. FileIO and metadata validation would remain outside the retry loop and run only once. Question: Should the CREATE -> UPDATE race condition (where CREATE fails because the entity already exists) be handled in the same change, or should that be a separate follow-up? Also, is there an existing retry utility or pattern in Polaris that should be reused? Location: runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java (sendNotificationForTableLike) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
