clintropolis commented on code in PR #19830:
URL: https://github.com/apache/druid/pull/19830#discussion_r3936847629
##########
docs/development/extensions-core/catalog.md:
##########
@@ -43,6 +43,177 @@ allowing queries to be more concise, and simpler to write.
This also allows the
written into a defined column of the table is consistent with that columns
definition, minimizing errors where unexpected
data is written into a particular column of the table.
+### SQL DDL
+
+Tables can be defined with SQL instead of by posting a table specification.
`CREATE TABLE` and `ALTER TABLE` are
+submitted to the Broker like any other SQL statement, and write the same
catalog metadata the REST API does. They
+return no rows.
+
+These statements change catalog metadata only. They never create, modify, or
delete segments: defining a table does
+not ingest anything, and altering a column does not rewrite existing data.
Column changes take effect for subsequent
+ingestion.
+
+These statements are disabled by default. Set
`druid.sql.planner.enableCatalogDdl` to `true` on the Broker to enable
+them. They require `WRITE` permission on the datasource, the same permission
the catalog API requires, so enabling
+them lets anyone who can ingest into a datasource also change its catalog
definition; leave them disabled if you
+manage catalog entries with your own tooling. The setting cannot be overridden
per query.
+
+The `druid-catalog` extension must be loaded on both the Broker and the
Coordinator; without it, these statements
+report that the extension is not available.
+
+```sql
+CREATE [OR REPLACE] TABLE [IF NOT EXISTS] <table>
+ [ ( { <column> <type> | PROJECTION <name> AS ( <select> ) } [, ...] ) ]
+ [ PARTITIONED BY <granularity> ]
+ [ CLUSTERED BY <column> [, ...] ]
+ [ SEALED ]
+```
+
+`OR REPLACE` replaces the specification of an existing table; `IF NOT EXISTS`
leaves an existing table unchanged.
+The two cannot be combined. `PARTITIONED BY` sets
[`segmentGranularity`](#table-properties) and `CLUSTERED BY` sets
+`clusterKeys`, both of which a later `INSERT` or `REPLACE` inherits unless it
states its own. `SEALED` sets
+[`sealed`](#table-properties), which requires every ingested column to be
declared.
+
+Note that the table-level `CLUSTERED BY` is a sort order applied to each
ingestion, which is a different thing from
+the `CLUSTERED BY` inside a [`__base` projection](#the-base-table), which
defines how segments physically group rows.
+
+Column types are written as SQL types, such as `VARCHAR`, `BIGINT`, `DOUBLE`,
or `VARCHAR ARRAY`. The `__time` column
+is written as `TIMESTAMP`. Types that have no SQL spelling, such as complex
types, use `TYPE('...')` with the Druid
+native type string:
+
+```sql
+CREATE TABLE "druid"."visits" (
+ __time TIMESTAMP,
+ user_id VARCHAR,
+ pages_visited BIGINT,
+ sketch TYPE('COMPLEX<thetaSketch>')
+)
+PARTITIONED BY DAY
+CLUSTERED BY user_id
+```
+
+`ALTER TABLE` supports one change per statement, so that each statement is a
single atomic catalog operation:
+
+```sql
+ALTER TABLE <table> ADD COLUMN <column> <type>
+ALTER TABLE <table> DROP COLUMN <column>
Review Comment:
documented that it is just a metadata operation and the implications, there
is `hiddenColumns` (which are not wired up in DDL in this PR), though on
thinking about it, it does feel kind of weird that `sealed` doesn't apply to
the schema, only ingest validation... (though not going to change that in this
PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
