dengc367 opened a new issue, #14309:
URL: https://github.com/apache/gravitino/issues/14309
## Describe the feature
Add three metalake-level metadata objects so Gravitino can describe business
semantics, not only store
definitions:
1. **Ontology** — an Apache Ossie `ontology.json` document: concepts
(`EntityType` / `ValueType`),
relationships with cardinality and natural-language phrasings, and IRIs.
2. **OntologyMapping** — an Ossie mapping document binding a Semantic
Model's datasets, fields and
relationships onto ontology concepts.
3. **Page** — a governed definition of a business term: description,
synonyms, Markdown body, related
assets, sources, owner, and draft/published state.
Design document: `design-docs/gravitino-ontology-design.md`, in the linked
pull request.
## Motivation
The Semantic Model design defers this explicitly:
> **Ontology management.** Apache Ossie Ontology definitions and mappings
are separate metadata
> concepts and require their own design.
Domains group assets by tag but define no concepts. Semantic Models hold
metrics and joins but do not
say what a concept means. Two questions therefore have no home today: *what
concepts exist and how do
they relate*, and *what does this term mean, and who vouches for it*.
The reference is Databricks Unity Catalog semantics, whose metric views,
domains, pages and
certification form the human-modeled layer of the Genie Ontology.
## Describe the solution
Ossie provides the concept graph but no narrative layer, so the design
separates two kinds of thing:
- **Concepts and mappings are machine-readable** and conform to Ossie
(`ontology.json`, and a
standalone mapping document, because Ossie deprecates embedding the
mapping in the ontology).
- **Pages are governed documents**, not Ossie concepts: Ossie's concept
`type` is a closed enum, and a
concept has no body, owner, synonyms or status.
No separate Glossary entity is proposed. The reference implementation
defines terms through Pages, and a
glossary over Pages is a listing rather than a second store.
Privileges are separate from Domains: `MANAGE_ONTOLOGY` / `USE_ONTOLOGY` and
`MANAGE_PAGE` / `USE_PAGE`,
and none of them grants access to data. Creating a mapping requires both
`USE_SEMANTIC_MODEL` on the
model and `USE_ONTOLOGY` on the ontology.
## Additional context
Phased: ontology, then mappings, then pages, then collaboration (comments
and suggestion batches), then
clients, MCP and UI.
The design document records six open questions. The two most consequential
are the scope of `Ontology`
(metalake versus schema, as Semantic Models are) and the placement of
`OntologyMapping` (a metalake
object versus a child of the Semantic Model).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
