xiangfu0 opened a new pull request, #18639: URL: https://github.com/apache/pinot/pull/18639
## Summary Makes `CREATE MATERIALIZED VIEW ... AS <query>` DDL pluggable via a new `MaterializedViewDdlHandler` extension point, so downstream distributions can support materialized views that are materialized by a different engine / minion task type (e.g. a multi-stage-engine MV whose `AS` clause is a JOIN) without forking the DDL compiler. OSS behavior is **byte-for-byte unchanged** when no alternative handler is registered: a JOIN in the `AS` clause is still rejected, and the MV is still routed under the built-in `MaterializedViewTask`. ## Motivation Today the single-source / single-stage assumption is hardcoded in `DdlCompiler.compileCreateMaterializedView`: - a JOIN in the `AS` clause is rejected unconditionally, and - the MV task config is always stamped under `MaterializedViewTask`. A distribution that wants a richer MV (e.g. materialize the result of a JOIN through the multi-stage engine via its own minion task type) has no clean seam — it cannot reuse the DDL compile path. Because the controller validates the resulting `TableConfig` (running the task generator's validation) *before* persisting, simply "allowing the JOIN at compile" is insufficient: the handler must be able to stamp a task type whose generator can validate that definition. ## What changed (`pinot-sql-ddl`) - **New SPI `MaterializedViewDdlHandler`** with two methods: `validateDefinedQuery(queryNode, properties)` (query/join policy, run before column resolution) and `applyTaskConfig(properties, definedSql, schedule, builder)` (routes the MV task config and returns the task type stamped). Includes a shared static `containsJoin(SqlNode)` helper. - **`DefaultMaterializedViewDdlHandler`** — preserves current behavior exactly (reject JOINs, route under `MaterializedViewTask`). - **`MaterializedViewDdlHandlerRegistry`** — process-wide registry, defaults to the single-source handler; a distribution registers its handler once at controller startup (mirrors existing pluggable-component registry patterns). - **`DdlCompiler.compileCreateMaterializedView`** delegates query validation + task routing to the registered handler, and validates MV consistency (`bucketTimePeriod` present) against the task type the handler stamped. - **`MaterializedViewPropertyRouter.apply`** gains a `taskType`-parameterized overload so a handler can route the MV task config under an alternative task type (the no-arg form delegates with `MaterializedViewTask`). - **`verifyDefinedSqlIsParseable`** now does a *syntactic* parse (`compileToSqlNodeAndOptions`) rather than full single-stage compilation (`compileToPinotQuery`). The slicing-bug guard only needs to confirm the extracted text is well-formed; the syntactic parse also accepts multi-stage shapes (e.g. JOINs) a handler may permit. Engine-specific validity is still enforced later by the MV analyzer / task generator for whichever task type the handler stamped. JOINs remain rejected by default (the default handler rejects them earlier, before this check). ## Testing - All existing `pinot-sql-ddl` tests pass (169), including `DdlCompilerMaterializedViewTest` (40) — default behavior (single-source, JOIN rejected, `MaterializedViewTask` routing) is unchanged. ## Backward compatibility No config, wire-format, or default-behavior changes. The extension point is opt-in via the registry; absent any registration, the default handler reproduces the prior code path. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
