yuqi1129 commented on code in PR #13867: URL: https://github.com/apache/gravitino/pull/13867#discussion_r4294586332
########## design-docs/domain-design.md: ########## @@ -0,0 +1,332 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Design: Domain + +| Version | Author | Date | Status | +| :---- | :---- | :---- | :---- | +| 1.26 | Jerry Shao | 2026-10-16 | Draft | + +## Goals + +1. **Business grouping**: group assets from any catalog under named domains (e.g. "Finance") for discovery, browsing and agent retrieval. An asset can belong to several domains. +2. **Membership is tags**: a leaf asset is a member when it carries the domain's backing tag, directly or inherited from its catalog or schema. There is no separate membership store. +3. **Separate privileges**: domain privileges govern the `Domain` object; tag privileges govern membership. Membership grants no access to data. +4. **One level of subdomains**, each with one parent; a subdomain can be moved. +5. **Draft and published status**: a `DRAFT` domain is returned only to its owner and managers, the same on REST, MCP and any UI. +6. **Inherited membership**: tagging a catalog or schema makes the leaf objects under it members, through Gravitino's existing tag inheritance. + +## Non-Goals + +1. **Approval workflow**: adding members, attaching subdomains and publishing take effect at once; status is a manual switch. +2. **Data-access grants on domains**: a domain carries no data privileges such as `SELECT_TABLE` (see Grants and policies). +3. **Domain-level policies**: domains add no policy mechanism; policies keep following tags (see Grants and policies). +4. **Deeper or multi-parent hierarchies**: the storage allows them later without a data migration. +5. **Automatic membership**, glossary and metric designs, and the ABAC policy language. Glossary terms can become members once they are taggable and added to the member types. +6. **UI**: this design covers the server, REST APIs, clients and MCP tools. + +## Usage Scenario + +1. A steward with `MANAGE_DOMAIN` and `CREATE_TAG` on the metalake creates domain Finance with a new backing tag `finance`. The steward owns both. +2. The steward creates subdomain Trading under Finance with a new backing tag `trading`, and grants `APPLY_TAG` on tag `trading` to the trading table owners. +3. A table owner tags `cat.db.trades` with `trading`. The table is a member of Trading, and is listed under Finance as "via subdomain Trading". Tagging schema `cat.db` with `finance` would make every leaf object in it a member of Finance. +4. A user or agent with `USE_DOMAIN` on Finance (which also covers Trading, unless denied there) or on Trading browses it and gets the tables they can access. +5. Trading is moved under Markets. Its members are now listed under Markets; nothing is retagged. + +## Design + +### Data model + +``` +Domain { id, name, comment, status: DRAFT | PUBLISHED, properties, + backingTagId?, backingTagValue?, audit_info } +DomainRel { parentId, childId, audit_info } // one edge per subdomain +``` + +- Domains live under a metalake. A subdomain and its parent are in the same metalake. +- Domain names are unique within a metalake across both levels: top-level domains and subdomains share one namespace, so a domain is addressed by its name alone. Domain names follow the same constraints as tag names (up to 128 characters) and may contain `/` (see REST API, Encoding). +- A unique index on `DomainRel (child_id, deleted_at)` keeps one parent per domain. `parent` is not stored on `Domain`; the API derives it. Dropping the index and checks would allow multiple parents later without a data migration. +- The backing tag is stored on the domain row as the tag's ID and an optional value, with an index on `(metalake_id, backing_tag_id, backing_tag_value, deleted_at)` for the reverse lookup "which domain does this (tag, value) back". `DOMAIN` is not a taggable object type, so the tag code paths are unchanged (see Alternatives considered). +- The binding refers to the tag by ID, and matching compares tag IDs: if the tag is deleted, the ID stays on the row and matches nothing, and a new tag with the same name does not match it. +- Domains, their parent and their backing tag are not cached; every request reads them from the store, so all servers see a move or a backing-tag change at once. Caching them later needs a version bump of the affected domain rows and a changelog entry. + +### Backing tag + +Each domain has **at most one** backing tag: an existing tag, optionally with a value (tag values: apache/gravitino#12162). Creating a domain always sets one; a domain has none only after its backing tag is removed or deleted, until a new one is set. + +- **Explicit at creation**: every create request names the backing tag; there is no default name derived from the domain. The caller either + - creates a new tag: a tag without values (`NO_VALUE`) with the given name, owned by the caller, which needs `CREATE_TAG`. If a tag with that name exists, creation fails with 409 `TagAlreadyExistsException`; an existing tag is never adopted implicitly; or + - binds an existing (tag, value), such as `dept=finance`. The caller must own the tag or the metalake; a missing tag fails with 404. + + A create request without a backing tag, or one that creates a new tag and also gives a value, fails with 400. +- **Tag value rules apply as for any assignment**: a tag with a non-empty allowed-values list can only be bound with one of its values; a `NO_VALUE` tag only without a value. +- **No sharing**: following the tag assignment rules, a tag has either one binding without a value, or bindings with values where each value backs at most one domain; the two are never mixed. A conflicting binding fails with 409 `BackingTagAlreadyBoundException` (a new `AlreadyExistsException` subclass). The reverse lookup is therefore one domain per (tag, value). +- **Matching** follows the tag lookup semantics (`GET /tags/{tag}/objects`). A backing tag without a value matches every assignment of the tag, with or without values, like a lookup without `value`. A backing tag with a value matches assignments whose values contain it, exact and case-sensitive, like `?value=`; a valueless assignment never matches it. The same rule applies to member listing and domains of an object, always against the object's effective assignment. +- **Changing it**: `PUT /domains/{domain}/backingTag` replaces or removes it on the server, under the domain and tag row locks. A missing tag fails with 404, a value the tag does not allow with 400, a tag the caller does not own (nor the metalake) with 403, and a binding that conflicts with another domain with 409; setting the current tag is a no-op. Assets carrying only the old tag drop out; nothing is migrated. This endpoint is the only way to change a backing tag: `DOMAIN` is not taggable, so the generic tag association API rejects it. +- **Never renamed**: moving or renaming a domain does not rename its tag; the hierarchy lives in `DomainRel`. A renamed tag keeps backing the domain, since the binding is by tag ID. +- **Deleting the tag** behaves as for any tag: it is allowed, and the tag's assignments on assets are removed with it. Its domain is left with no backing tag (`backingTag` reads as empty) and no members until a new one is set; the membership cannot be recovered by recreating a tag with the same name. Only the tag deletion event records this. +- **Separate from domain privileges**: the domain owner and the tag owner are independent. Managing a domain grants no `APPLY_TAG` on its tag, and transferring the domain's ownership does not transfer the tag's. + +### Subdomains + +Every operation that sets a parent (create a subdomain, attach, move) checks: + +| # | Constraint | Review Comment: Better to align the table to make it more human-readable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
