roryqi opened a new pull request, #11362: URL: https://github.com/apache/gravitino/pull/11362
### What changes were proposed in this pull request? This PR cleans up Gravitino schema entities that become orphaned when the underlying catalog automatically removes empty parent namespaces after dropping a table, view, or namespace. The patch: - shares orphan schema cleanup logic in the core operation dispatcher - applies the cleanup from schema, table, and view drop paths - adds Iceberg REST schema entity cleanup for namespace, table, and view drops - aligns Iceberg REST table/view drop cleanup with the namespace hook branch lock - adds regression tests for core dispatchers and Iceberg REST hook dispatchers ### Why are the changes needed? Some catalogs, including Iceberg JdbcCatalog, may implicitly remove empty namespaces and their empty parent namespaces after the last table/view/sub-namespace is dropped. Gravitino could still keep the corresponding schema entities in its store, leaving stale metadata. Fix: #11262 ### Does this PR introduce _any_ user-facing change? No new API, configuration, or user-facing behavior is introduced. This fixes stale internal metadata cleanup after drop operations. ### How was this patch tested? - Skipping filesystem-fuse module since enableFuse is set to false > Configure project : ------------------ Check Docker environment --------------------- Docker server status ............................................ [running] Run test cases without `gravitino-docker-test` tag .............. [embedded test] ----------------------------------------------------------------- > Configure project :catalogs Path for java installation '/usr/lib/jvm/openjdk-17' (Common Linux Locations) does not contain a java executable > Configure project :core Kotlin DSL property assignment is an incubating feature. > Task :spotlessInternalRegisterDependencies UP-TO-DATE > Task :api:processResources UP-TO-DATE > Task :catalogs:catalog-common:processResources NO-SOURCE > Task :common:processResources > Task :catalogs:catalog-common:spotlessJava UP-TO-DATE > Task :catalogs:catalog-common:spotlessJavaCheck UP-TO-DATE > Task :catalogs:catalog-common:spotlessKotlinGradle UP-TO-DATE > Task :catalogs:catalog-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :catalogs:catalog-common:spotlessCheck UP-TO-DATE > Task :integration-test-common:spotlessJava UP-TO-DATE > Task :integration-test-common:spotlessJavaCheck UP-TO-DATE > Task :common:spotlessJava UP-TO-DATE > Task :common:spotlessJavaCheck UP-TO-DATE > Task :integration-test-common:spotlessKotlinGradle UP-TO-DATE > Task :integration-test-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :integration-test-common:spotlessCheck UP-TO-DATE > Task :common:spotlessKotlinGradle UP-TO-DATE > Task :common:spotlessKotlinGradleCheck UP-TO-DATE > Task :common:spotlessCheck UP-TO-DATE > Task :api:spotlessJava UP-TO-DATE > Task :integration-test-common:compileJava NO-SOURCE > Task :integration-test-common:processResources NO-SOURCE > Task :api:spotlessJavaCheck UP-TO-DATE > Task :integration-test-common:classes UP-TO-DATE > Task :api:spotlessKotlinGradle UP-TO-DATE > Task :api:spotlessKotlinGradleCheck UP-TO-DATE > Task :api:spotlessCheck UP-TO-DATE > Task :lineage:spotlessJava UP-TO-DATE > Task :lineage:spotlessJavaCheck UP-TO-DATE > Task :lineage:spotlessKotlinGradle UP-TO-DATE > Task :lineage:spotlessKotlinGradleCheck UP-TO-DATE > Task :lineage:spotlessCheck UP-TO-DATE > Task :lineage:processResources NO-SOURCE > Task :server-common:spotlessJava UP-TO-DATE > Task :server-common:spotlessJavaCheck UP-TO-DATE > Task :server-common:spotlessKotlinGradle UP-TO-DATE > Task :server-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :server-common:spotlessCheck UP-TO-DATE > Task :server-common:processResources UP-TO-DATE > Task :server:spotlessJava UP-TO-DATE > Task :clients:client-java:spotlessJava UP-TO-DATE > Task :server:spotlessJavaCheck UP-TO-DATE > Task :clients:client-java:spotlessJavaCheck UP-TO-DATE > Task :server:spotlessKotlinGradle UP-TO-DATE > Task :clients:client-java:spotlessKotlinGradle UP-TO-DATE > Task :server:spotlessKotlinGradleCheck UP-TO-DATE > Task :server:spotlessCheck UP-TO-DATE > Task :clients:client-java:spotlessKotlinGradleCheck UP-TO-DATE > Task :clients:client-java:spotlessCheck UP-TO-DATE > Task :server:processResources NO-SOURCE > Task :clients:client-java:processResources NO-SOURCE > Task :api:compileJava UP-TO-DATE > Task :api:classes UP-TO-DATE > Task :integration-test-common:processTestResources UP-TO-DATE > Task :api:jar UP-TO-DATE > Task :lance:lance-common:spotlessJava UP-TO-DATE > Task :lance:lance-common:spotlessJavaCheck UP-TO-DATE > Task :lance:lance-common:spotlessKotlinGradle UP-TO-DATE > Task :lance:lance-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :lance:lance-common:spotlessCheck UP-TO-DATE > Task :lance:lance-common:processResources NO-SOURCE > Task :common:compileJava UP-TO-DATE > Task :common:classes > Task :core:spotlessJava UP-TO-DATE > Task :core:spotlessJavaCheck UP-TO-DATE > Task :core:spotlessKotlinGradle UP-TO-DATE > Task :core:spotlessKotlinGradleCheck UP-TO-DATE > Task :core:spotlessCheck UP-TO-DATE > Task :core:processResources UP-TO-DATE > Task :core:processTestResources UP-TO-DATE > Task :common:jar WARN: Unable to get Git commit id : /home/roryqi/code/gravitino/.claude/worktrees/fix-11262-drop-schema-hierarchical/home/roryqi/code/gravitino/.git/worktrees/fix-11262-drop-schema-hierarchical/HEAD (No such file or directory) > Task :clients:client-java:compileJava UP-TO-DATE > Task :clients:client-java:classes UP-TO-DATE > Task :clients:client-java:jar UP-TO-DATE > Task :catalogs:catalog-common:compileJava UP-TO-DATE > Task :catalogs:catalog-common:classes UP-TO-DATE > Task :catalogs:catalog-common:jar UP-TO-DATE > Task :core:compileJava UP-TO-DATE > Task :core:classes UP-TO-DATE > Task :core:jar UP-TO-DATE > Task :lance:lance-common:compileJava UP-TO-DATE > Task :lance:lance-common:classes UP-TO-DATE > Task :server-common:compileJava UP-TO-DATE > Task :server-common:classes UP-TO-DATE > Task :lance:lance-common:jar UP-TO-DATE > Task :server-common:jar UP-TO-DATE > Task :lineage:compileJava UP-TO-DATE > Task :lineage:classes UP-TO-DATE > Task :lineage:jar UP-TO-DATE > Task :server:compileJava UP-TO-DATE > Task :server:classes UP-TO-DATE > Task :server:jar UP-TO-DATE > Task :integration-test-common:compileTestJava UP-TO-DATE > Task :integration-test-common:testClasses UP-TO-DATE > Task :integration-test-common:testJar UP-TO-DATE > Task :core:compileTestJava UP-TO-DATE > Task :core:testClasses UP-TO-DATE > Task :core:test [TEST-TIMING] START module=:core task=:core:test at=1780398228093 [TEST-TIMING] END module=:core task=:core:test startMs=1780398228093 endMs=1780398238166 durationMs=10073 > Task :core:jacocoTestReport Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. BUILD SUCCESSFUL in 22s 70 actionable tasks: 4 executed, 66 up-to-date - Skipping filesystem-fuse module since enableFuse is set to false > Configure project : ------------------ Check Docker environment --------------------- Docker server status ............................................ [running] Run test cases without `gravitino-docker-test` tag .............. [embedded test] ----------------------------------------------------------------- Path for java installation '/usr/lib/jvm/openjdk-17' (Common Linux Locations) does not contain a java executable > Configure project :core Kotlin DSL property assignment is an incubating feature. > Task :catalogs:catalog-common:processResources NO-SOURCE > Task :spotlessInternalRegisterDependencies UP-TO-DATE > Task :api:processResources UP-TO-DATE > Task :server-common:processResources UP-TO-DATE > Task :common:processResources > Task :catalogs:catalog-common:spotlessJava UP-TO-DATE > Task :catalogs:catalog-common:spotlessJavaCheck UP-TO-DATE > Task :catalogs:catalog-common:spotlessKotlinGradle UP-TO-DATE > Task :catalogs:catalog-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :catalogs:catalog-common:spotlessCheck UP-TO-DATE > Task :server-common:spotlessJava UP-TO-DATE > Task :server-common:spotlessJavaCheck UP-TO-DATE > Task :api:spotlessJava UP-TO-DATE > Task :server-common:spotlessKotlinGradle UP-TO-DATE > Task :server-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :server-common:spotlessCheck UP-TO-DATE > Task :api:spotlessJavaCheck UP-TO-DATE > Task :api:spotlessKotlinGradle UP-TO-DATE > Task :api:spotlessKotlinGradleCheck UP-TO-DATE > Task :api:spotlessCheck UP-TO-DATE > Task :common:spotlessJava UP-TO-DATE > Task :bundles:aliyun:spotlessJava UP-TO-DATE > Task :common:spotlessJavaCheck UP-TO-DATE > Task :bundles:aliyun:spotlessJavaCheck UP-TO-DATE > Task :common:spotlessKotlinGradle UP-TO-DATE > Task :bundles:aliyun:spotlessKotlinGradle UP-TO-DATE > Task :bundles:aliyun:spotlessKotlinGradleCheck UP-TO-DATE > Task :common:spotlessKotlinGradleCheck UP-TO-DATE > Task :bundles:aliyun:spotlessCheck UP-TO-DATE > Task :common:spotlessCheck UP-TO-DATE > Task :bundles:aliyun:processResources UP-TO-DATE > Task :catalogs:hadoop-common:spotlessJava UP-TO-DATE > Task :catalogs:hadoop-common:spotlessJavaCheck UP-TO-DATE > Task :catalogs:hadoop-common:spotlessKotlinGradle UP-TO-DATE > Task :catalogs:hadoop-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :catalogs:hadoop-common:spotlessCheck UP-TO-DATE > Task :catalogs:hadoop-common:processResources UP-TO-DATE > Task :bundles:aws:spotlessJava UP-TO-DATE > Task :api:compileJava UP-TO-DATE > Task :api:classes UP-TO-DATE > Task :bundles:azure:spotlessJava UP-TO-DATE > Task :bundles:azure:spotlessJavaCheck UP-TO-DATE > Task :bundles:aws:spotlessJavaCheck UP-TO-DATE > Task :bundles:aws:spotlessKotlinGradle UP-TO-DATE > Task :bundles:azure:spotlessKotlinGradle UP-TO-DATE > Task :bundles:azure:spotlessKotlinGradleCheck UP-TO-DATE > Task :bundles:azure:spotlessCheck UP-TO-DATE > Task :bundles:aws:spotlessKotlinGradleCheck UP-TO-DATE > Task :api:jar UP-TO-DATE > Task :bundles:aws:spotlessCheck UP-TO-DATE > Task :bundles:azure:processResources UP-TO-DATE > Task :bundles:aws:processResources UP-TO-DATE > Task :bundles:gcp:spotlessJava UP-TO-DATE > Task :bundles:gcp:spotlessJavaCheck UP-TO-DATE > Task :clients:client-java:spotlessJava UP-TO-DATE > Task :common:compileJava UP-TO-DATE > Task :common:classes > Task :clients:client-java:spotlessJavaCheck UP-TO-DATE > Task :bundles:gcp:spotlessKotlinGradle UP-TO-DATE > Task :bundles:gcp:spotlessKotlinGradleCheck UP-TO-DATE > Task :clients:client-java:spotlessKotlinGradle UP-TO-DATE > Task :clients:client-java:spotlessKotlinGradleCheck UP-TO-DATE > Task :bundles:gcp:spotlessCheck UP-TO-DATE > Task :bundles:gcp:processResources UP-TO-DATE > Task :clients:client-java:spotlessCheck UP-TO-DATE > Task :clients:client-java:processResources NO-SOURCE > Task :iceberg:iceberg-common:spotlessJava UP-TO-DATE > Task :iceberg:iceberg-common:spotlessJavaCheck UP-TO-DATE > Task :iceberg:iceberg-common:spotlessKotlinGradle UP-TO-DATE > Task :iceberg:iceberg-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :iceberg:iceberg-common:spotlessCheck UP-TO-DATE > Task :iceberg:iceberg-common:processResources NO-SOURCE > Task :integration-test-common:spotlessJava UP-TO-DATE > Task :iceberg:iceberg-rest-server:spotlessJava UP-TO-DATE > Task :integration-test-common:spotlessJavaCheck UP-TO-DATE > Task :iceberg:iceberg-rest-server:spotlessJavaCheck UP-TO-DATE > Task :integration-test-common:spotlessKotlinGradle UP-TO-DATE > Task :integration-test-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :integration-test-common:spotlessCheck UP-TO-DATE > Task :integration-test-common:compileJava NO-SOURCE > Task :integration-test-common:processResources NO-SOURCE > Task :iceberg:iceberg-rest-server:spotlessKotlinGradle UP-TO-DATE > Task :integration-test-common:classes UP-TO-DATE > Task :iceberg:iceberg-rest-server:spotlessKotlinGradleCheck UP-TO-DATE > Task :iceberg:iceberg-rest-server:spotlessCheck UP-TO-DATE > Task :iceberg:iceberg-rest-server:processResources UP-TO-DATE > Task :lineage:spotlessJava UP-TO-DATE > Task :server:spotlessJava UP-TO-DATE > Task :server:spotlessJavaCheck UP-TO-DATE > Task :lineage:spotlessJavaCheck UP-TO-DATE > Task :server:spotlessKotlinGradle UP-TO-DATE > Task :lineage:spotlessKotlinGradle UP-TO-DATE > Task :lineage:spotlessKotlinGradleCheck UP-TO-DATE > Task :server:spotlessKotlinGradleCheck UP-TO-DATE > Task :lineage:spotlessCheck UP-TO-DATE > Task :server:spotlessCheck UP-TO-DATE > Task :lineage:processResources NO-SOURCE > Task :server:processResources NO-SOURCE > Task :integration-test-common:processTestResources UP-TO-DATE > Task :bundles:iceberg-aliyun-bundle:spotlessJava UP-TO-DATE > Task :bundles:iceberg-aliyun-bundle:spotlessJavaCheck UP-TO-DATE > Task :bundles:iceberg-aliyun-bundle:spotlessKotlinGradle UP-TO-DATE > Task :lance:lance-common:spotlessJava UP-TO-DATE > Task :bundles:iceberg-aliyun-bundle:spotlessKotlinGradleCheck UP-TO-DATE > Task :bundles:iceberg-aliyun-bundle:spotlessCheck UP-TO-DATE > Task :lance:lance-common:spotlessJavaCheck UP-TO-DATE > Task :lance:lance-common:spotlessKotlinGradle UP-TO-DATE > Task :bundles:iceberg-aliyun-bundle:compileJava NO-SOURCE > Task :bundles:iceberg-aliyun-bundle:processResources NO-SOURCE > Task :bundles:iceberg-aliyun-bundle:classes UP-TO-DATE > Task :lance:lance-common:spotlessKotlinGradleCheck UP-TO-DATE > Task :lance:lance-common:spotlessCheck UP-TO-DATE > Task :lance:lance-common:processResources NO-SOURCE > Task :bundles:iceberg-aws-bundle:spotlessJava UP-TO-DATE > Task :bundles:iceberg-aws-bundle:spotlessJavaCheck UP-TO-DATE > Task :bundles:iceberg-aws-bundle:spotlessKotlinGradle UP-TO-DATE > Task :bundles:iceberg-aliyun-bundle:shadowJar UP-TO-DATE > Task :bundles:iceberg-aws-bundle:spotlessKotlinGradleCheck UP-TO-DATE > Task :bundles:iceberg-aws-bundle:spotlessCheck UP-TO-DATE > Task :bundles:iceberg-aws-bundle:compileJava NO-SOURCE > Task :bundles:iceberg-aliyun-bundle:jar UP-TO-DATE > Task :bundles:iceberg-aws-bundle:processResources NO-SOURCE > Task :bundles:iceberg-aws-bundle:classes UP-TO-DATE > Task :bundles:iceberg-azure-bundle:spotlessJava UP-TO-DATE > Task :common:jar WARN: Unable to get Git commit id : /home/roryqi/code/gravitino/.claude/worktrees/fix-11262-drop-schema-hierarchical/home/roryqi/code/gravitino/.git/worktrees/fix-11262-drop-schema-hierarchical/HEAD (No such file or directory) > Task :bundles:iceberg-aws-bundle:shadowJar UP-TO-DATE > Task :core:spotlessJava UP-TO-DATE > Task :core:spotlessJavaCheck UP-TO-DATE > Task :core:spotlessKotlinGradle UP-TO-DATE > Task :core:spotlessKotlinGradleCheck UP-TO-DATE > Task :core:spotlessCheck UP-TO-DATE > Task :core:processResources UP-TO-DATE > Task :bundles:iceberg-aws-bundle:jar UP-TO-DATE > Task :bundles:iceberg-azure-bundle:spotlessJavaCheck UP-TO-DATE > Task :catalogs:catalog-common:compileJava UP-TO-DATE > Task :catalogs:catalog-common:classes UP-TO-DATE > Task :bundles:iceberg-azure-bundle:spotlessKotlinGradle UP-TO-DATE > Task :bundles:iceberg-azure-bundle:spotlessKotlinGradleCheck UP-TO-DATE > Task :bundles:iceberg-azure-bundle:spotlessCheck UP-TO-DATE > Task :bundles:iceberg-azure-bundle:compileJava NO-SOURCE > Task :clients:client-java:compileJava UP-TO-DATE > Task :clients:client-java:classes UP-TO-DATE > Task :bundles:iceberg-azure-bundle:processResources NO-SOURCE > Task :bundles:iceberg-azure-bundle:classes UP-TO-DATE > Task :catalogs:catalog-common:jar UP-TO-DATE > Task :clients:client-java:jar UP-TO-DATE > Task :bundles:iceberg-azure-bundle:shadowJar UP-TO-DATE > Task :catalogs:hadoop-common:compileJava UP-TO-DATE > Task :catalogs:hadoop-common:classes UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:spotlessJava UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:spotlessJavaCheck UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:spotlessKotlinGradle UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:spotlessKotlinGradleCheck UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:spotlessCheck UP-TO-DATE > Task :bundles:iceberg-azure-bundle:jar UP-TO-DATE > Task :catalogs:hadoop-common:jar UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:compileJava NO-SOURCE > Task :iceberg:iceberg-rest-server:processTestResources UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:processResources NO-SOURCE > Task :bundles:iceberg-gcp-bundle:classes UP-TO-DATE > Task :catalogs:catalog-fileset:runtimeJars UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:shadowJar UP-TO-DATE > Task :bundles:aliyun:compileJava UP-TO-DATE > Task :bundles:aliyun:classes UP-TO-DATE > Task :bundles:aws:compileJava UP-TO-DATE > Task :bundles:aws:classes UP-TO-DATE > Task :bundles:aliyun:jar UP-TO-DATE > Task :bundles:aws:jar UP-TO-DATE > Task :core:compileJava UP-TO-DATE > Task :bundles:iceberg-gcp-bundle:jar UP-TO-DATE > Task :core:classes UP-TO-DATE > Task :bundles:azure:compileJava UP-TO-DATE > Task :bundles:azure:classes UP-TO-DATE > Task :bundles:gcp:compileJava UP-TO-DATE > Task :bundles:gcp:classes UP-TO-DATE > Task :bundles:azure:jar UP-TO-DATE > Task :core:jar UP-TO-DATE > Task :bundles:gcp:jar UP-TO-DATE > Task :lance:lance-common:compileJava UP-TO-DATE > Task :lance:lance-common:classes UP-TO-DATE > Task :server-common:compileJava UP-TO-DATE > Task :lance:lance-common:jar UP-TO-DATE > Task :server-common:classes UP-TO-DATE > Task :iceberg:iceberg-common:compileJava UP-TO-DATE > Task :server-common:jar UP-TO-DATE > Task :iceberg:iceberg-common:classes UP-TO-DATE > Task :iceberg:iceberg-common:jar UP-TO-DATE > Task :lineage:compileJava UP-TO-DATE > Task :lineage:classes UP-TO-DATE > Task :iceberg:iceberg-rest-server:compileJava UP-TO-DATE > Task :lineage:jar UP-TO-DATE > Task :iceberg:iceberg-rest-server:classes UP-TO-DATE > Task :iceberg:iceberg-rest-server:jar UP-TO-DATE > Task :server:compileJava UP-TO-DATE > Task :server:classes UP-TO-DATE > Task :server:jar UP-TO-DATE > Task :integration-test-common:compileTestJava UP-TO-DATE > Task :integration-test-common:testClasses UP-TO-DATE > Task :integration-test-common:testJar UP-TO-DATE > Task :iceberg:iceberg-rest-server:copyDepends > Task :iceberg:iceberg-rest-server:compileTestJava UP-TO-DATE > Task :iceberg:iceberg-rest-server:testClasses UP-TO-DATE > Task :iceberg:iceberg-rest-server:test [TEST-TIMING] START module=:iceberg:iceberg-rest-server task=:iceberg:iceberg-rest-server:test at=1780398251143 [TEST-TIMING] END module=:iceberg:iceberg-rest-server task=:iceberg:iceberg-rest-server:test startMs=1780398251143 endMs=1780398258494 durationMs=7351 > Task :iceberg:iceberg-rest-server:jacocoTestReport Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. BUILD SUCCESSFUL in 16s 144 actionable tasks: 5 executed, 139 up-to-date - -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
