hs150 opened a new pull request, #13757: URL: https://github.com/apache/gravitino/pull/13757
[#13733] fix(catalog): Remove TreeLock from external-catalog operations What changes were proposed in this pull request? This pull request addresses issue #13733 by removing TreeLock-based synchronization from external-catalog entity operations and the catalog drop lifecycle. The changes include: - Removing TreeLock wrappers from schema, table, topic, and view operation dispatchers. - Removing TreeLock from `CatalogManager.dropCatalog()`. - Introducing identity-fenced update and delete operations to prevent stale operations from modifying or deleting concurrently replaced entities. - Fixing a cache publication race that could allow stale data to be repopulated after cache invalidation. - Strengthening transactional lifecycle checks to protect parent-child consistency during catalog and metalake deletion. - Adding concurrency regression tests for multi-node operations, concurrent drops, identity fencing, and late cache publication. - Updating existing dispatcher and memory entity-store tests. ### Why are the changes needed? TreeLock-based synchronization is process-local and cannot coordinate concurrent operations across multiple Gravitino server instances. Without appropriate concurrency controls, concurrent updates, deletions, or entity replacements can result in stale operations affecting newer entities. A late cache fill can also publish stale data after invalidation. This patch uses database transactions, identity/version fencing, and cache invalidation coordination to address these concurrency issues without relying on TreeLock for the affected operations. ### Does this PR introduce any user-facing change? No intentional user-facing API changes are introduced. The changes are focused on internal synchronization, metadata consistency, cache behavior, and concurrency correctness. ### How was this patch tested? The following tests and checks passed locally: - `TestMultiNodeConcurrency` - `TestCatalogManager` - `TestMetalakeManager` - `TestExternalEntityIdentityFencingAndLifecycle` - Operation dispatcher tests - `TestMemoryEntityStore` - `:core:spotlessCheck` - `git diff --check` - `git diff --cached --check` Regression tests cover concurrent entity operations across multiple store instances, identity-fenced updates and deletions, lifecycle preconditions, and stale cache publication. Fix: #13733 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
