dengc367 opened a new issue, #14308:
URL: https://github.com/apache/gravitino/issues/14308
## Describe the feature
Add a **query planning contract** to Semantic Model: a caller can ask
Gravitino to turn a semantic
query (metrics, dimensions, filters) into SQL for a given engine, and
Gravitino returns the SQL
without executing it.
Three pieces:
1. **REST endpoint**
```
POST
/metalakes/{metalake}/catalogs/{catalog}/schemas/{schema}/semantic-models/{semanticModel}/plan
```
Request:
```json
{
"metrics": ["impressions", "ctr"],
"dimensions": ["page"],
"filters": [{ "field": "dt", "operator": "eq", "value": "2026-10-01" }],
"dialect": "SPARK"
}
```
Response:
```json
{
"sql": "SELECT ...",
"dialect": "SPARK",
"planner": "orionbelt",
"warnings": []
}
```
Errors: `400` malformed query, `403` not authorized, `404` model missing,
`422` query cannot be
compiled, `501` no planner configured.
Version pinning is deliberately left out of the first cut. The storage
layer already keeps
version snapshots (`semantic_model_version_info`), but the public API
exposes no version selector
today, so `plan` uses the current definition. See the open questions.
2. **Java SPI**
```java
public interface SemanticQueryPlanner extends Closeable {
String name();
SemanticQueryPlan plan(
SemanticModel model, SemanticQuery query, PlanningContext context);
}
```
Loaded like the existing plugin configs:
`gravitino.semanticModel.queryPlanner` selects a name,
`gravitino.semanticModelQueryPlanner.<name>.class` names the class.
Built-in default is `none`,
which makes `plan` answer `501`.
3. **MCP tool and clients**
- MCP: `compile_semantic_query` under the existing `semantic_model` tool
tag.
- Java: `default SemanticQueryPlan planSemanticQuery(NameIdentifier
ident, SemanticQuery query)`
on `SemanticModelCatalog`, throwing `UnsupportedOperationException`
when no planner is
configured — the same way `importOssieDocument` / `exportOssieDocument`
declare an optional
capability.
- Python: the same on the semantic model operations.
What Gravitino owns: request validation, authorization
(`USE_SEMANTIC_MODEL`), model resolution, the
response envelope, and events/audit.
What the planner owns: SQL generation, joins, metric fan-out rules, and the
dialect matrix.
Explicitly **not** included: execution, result fetching, materialization, a
bundled compiler, or
server-side plan caching.
## Motivation
A Semantic Model today is a governed definition. It cannot answer a question
about itself.
Every consumer therefore rebuilds the same pipeline: export the Ossie
document, parse it, resolve
metrics and dimensions, resolve join paths, handle fan-out, and emit dialect
SQL. That logic is
substantial and drifts between consumers, so two tools can return different
numbers for
`"ctr" by "page"` from the same model.
This matters more as the ecosystem grows:
- **Agents.** An LLM writing SQL against raw tables joins two fact tables
and inflates a metric,
and the error is invisible. A `plan` call gives it the model's own
definition and one correct
query.
- **BI tools and engines.** Each one needs the same compilation. Without a
shared contract, each one
ports the compiler or stays unsupported.
- **Governance.** When compilation happens outside the server, per-user
authorization holds only if
the consumer forwards the caller's credentials. A server-side endpoint
makes model resolution and
authorization a single auditable point, using privileges that already
exist.
## Describe the solution
### Relationship to the design's Non-Goals
The [Semantic Model
design](https://github.com/apache/gravitino/blob/main/design-docs/gravitino-semantic-model-design.md)
lists:
> **Query compilation and execution.** Semantic query planning, SQL
generation, engine execution,
> and engine-specific query syntax are separate work.
This proposal keeps that stance: **Gravitino still does not generate SQL**.
It defines the *seam*
and delegates the work through an SPI, so the work stays separate but its
contract is standardized
instead of reinvented per consumer.
A built-in compiler would require changing that non-goal and taking on the
dialect matrix; that is a
much larger scope and should be a separate discussion. The SPI leaves room
for it later.
### Why an SPI rather than a bundled implementation
The hard part is not the endpoint; it is the dialect matrix and planner
semantics. Bundling it would
make Gravitino own per-engine SQL correctness for every supported dialect.
An SPI also keeps non-Java compilers viable: an implementation can be a thin
in-process adapter that
calls an external compiler service. Gravitino then owns the contract, while
the compiler stays where
its dialect expertise is.
### Precedent in-tree
The shape follows existing extension points: `JobExecutor` and
`JobTemplateProvider` SPI, and the
catalog provider SPIs. Loading is the same two-key pattern
(`<feature>.<slot>` selects a name,
`<feature><Slot>.<name>.class` names the class).
### Open questions
1. Should the response carry SQL text, a structured relational plan, or both?
2. Is `dialect` a fixed enum or a free string?
3. Should version pinning be a path segment
(`.../semantic-models/{semanticModel}/versions/{version}/plan`) once the
API exposes versions,
rather than a body field?
4. Should `compile_semantic_query` join the read-only `semantic_model` tag,
or have its own tag?
## Additional context
- Correlated work: the design's future `Ontology` entity (currently a
Non-Goal) would let a planner
resolve business concepts, but it is not a prerequisite for this contract.
- Scope boundary for reviewers: this is about *compilation*, not
*execution*. The server never
connects to an engine, never returns rows, and never caches a plan.
- I am happy to prototype the endpoint and SPI behind a config flag to test
the contract against a
real compiler before committing to a public API shape.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
