LiJie20190102 opened a new pull request, #13431: URL: https://github.com/apache/gravitino/pull/13431
### What changes were proposed in this pull request? Add a design document for reconciling stale registrations of non-managed entities with their source: - Define "stale" for a registration: the owning catalog does not manage storage for the entity scope, an explicit per-object existence probe reports the object absent, and the probe did not fail for infrastructural reasons (unreachable sources are skipped, never treated as absent). - Propose a server-side `StaleSchemaReconciler` following the `RelationalGarbageCollector` pattern, with a periodic trigger and an on-demand REST API, and a report-only / auto-remove policy switch (default: disabled, report-only when enabled). - Removal goes through `SchemaDispatcher` under the server's tree lock, so events, secret cleanup, authorization-plugin privilege removal and orphan cleanup behave exactly as an explicit cascading drop (reusing the #13279 semantics). - Record open questions for community discussion: rename handling via `gravitino.identifier`, system identity for the background task, report caching, per-catalog policy overrides, and removal rate limiting. The first phase covers schemas only; tables, views, topics and filesets are left to follow-up tasks in the epic. ### Why are the changes needed? For non-managed catalogs (JDBC, Hive, Iceberg, Kafka, ...), objects dropped or renamed directly in the source leave stale registrations in Gravitino's store. Catalog-walking consumers (metrics, lineage, search sync) resolve these names and then fail on load with 404. Existing ad-hoc cleanups bypass the dispatcher chain and skip secret cleanup, privilege removal and events. This design doc aligns the community on stale semantics, safe absence confirmation, and trigger/policy options before implementation starts. Fix: #13429 Part of: #13313 ### Does this PR introduce _any_ user-facing change? No. Design document only. ### How was this patch tested? Documentation only; no code changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
