xiangfu0 opened a new pull request, #18723:
URL: https://github.com/apache/pinot/pull/18723
## Description
Adds a second form to the `CREATE TABLE` SQL DDL, with no column list and no
`TABLE_TYPE`:
```sql
CREATE TABLE [IF NOT EXISTS] [db.]name WITH (key = value, ...)
```
In this form the schema and table config are derived entirely from the WITH
options by a pluggable `CreateTableWithOptionsHandler`, installed once at
controller startup via `DdlCompiler#setCreateTableWithOptionsHandler` —
mirroring the existing `MaterializedViewDdlHandler` extension point. OSS Pinot
has no built-in options-defined table type, so the default handler rejects the
form with actionable guidance pointing at the column-list form; behavior of
existing deployments is unchanged.
The motivating use case is external-table DDL, e.g.:
```sql
CREATE TABLE trips_analytics WITH (
type = 'iceberg',
catalog_type = 'rest',
catalog_uri =
'https://unity-catalog.company.com/api/2.1/unity-catalog/iceberg',
schema_name = 'transportation',
table_name = 'nyc_taxi_trips',
storage.region = 'us-west-2',
refresh_interval = '5m',
enable_schema_evolution = true
)
```
where a distribution-specific handler connects to the external catalog named
by the options, infers the Pinot schema from the catalog table, and emits a
`TableConfig` wired to its ingestion machinery. The compiled
`CompiledCreateTable` flows through the existing `POST /sql/ddl` execution path
(authorization, validation, persistence, `IF NOT EXISTS`) with no controller
changes.
### Grammar
- The two CREATE TABLE forms are mutually exclusive, disambiguated by the
token after the table name (`WITH` vs the column-list paren) — no lookahead
conflicts.
- WITH option keys may be quoted string literals or unquoted (possibly
dotted) identifiers (`catalog_type`, `storage.region`); values may be string
literals, `TRUE`/`FALSE`, or unsigned numeric literals. Everything normalizes
to string key/value pairs (`SqlPinotProperty`), so consumers uniformly see
ordered string maps and the existing case-insensitive duplicate-key detection
applies.
- `${...}` placeholder values pass through verbatim as opaque strings (no
parser-side substitution).
### Compiler contract
- `DdlCompiler` owns the parts common to both forms: qualified-name
resolution and option de-duplication; the handler owns what the options mean.
- Empty option lists and null handler results are rejected with clear errors.
### Testing
- `PinotDdlParserTest`: 9 new cases — minimal/full external-table sample, IF
NOT EXISTS + qualified names, key quoting and value types, empty list,
mutual-exclusivity rejections, unparse round-trip (60/60 pass).
- New `CreateTableWithOptionsHandlerTest`: default rejects with guidance,
registered handler receives parsed name/flags/ordered options, empty/duplicate
options rejected before the handler, null result rejected, column-list form
bypasses the handler (8/8 pass; full pinot-sql-ddl suite 184/184).
Suggested labels: `feature`, `extension-point`, `release-notes` (new SQL
syntax + new public extension interface).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
