jiangxt2 opened a new pull request, #12274:
URL: https://github.com/apache/gravitino/pull/12274
### What changes were proposed in this pull request?
Read the `engine_full` column from `system.tables` (already fetched via
`SELECT *`)
to extract engine parameters for non-Distributed MergeTree-family engines,
store
them as a new `engine_parameters` table property, and include them when
generating
`ENGINE = ...` clauses in `CREATE TABLE` DDL.
### Why are the changes needed?
The `engine` column in `system.tables` only returns the engine name (e.g.
`ReplacingMergeTree`) without parameters. After a create→load round-trip (or
loading
a pre-existing table), the parameters in parentheses are silently dropped.
`ReplacingMergeTree(ts)` becomes plain `ReplacingMergeTree` — the
deduplication
version column is lost, changing the table's correctness semantics.
This affects five MergeTree-family engines:
- `ReplacingMergeTree` — version column for dedup
- `CollapsingMergeTree` — sign column for collapse
- `VersionedCollapsingMergeTree` — sign + version for versioned collapse
- `SummingMergeTree` — columns (including nested-paren tuples) for auto-sum
- `GraphiteMergeTree` — graphite rollup config
The `engine_full` column already contains the full engine DDL and is already
parsed for `Distributed` engines; extending this to other engines reuses the
existing query path.
Fixes #12273
### Does this PR introduce _any_ user-facing change?
A new optional table property `engine_parameters` is added for ClickHouse
tables. Users can set it when creating tables (e.g.
`engine_parameters = "ts"` for `ReplacingMergeTree(ts)`), and it will appear
in `table.properties()` for loaded tables that have engine parameters.
No existing property keys are changed or removed.
### How was this patch tested?
- Unit tests: 9 test cases covering single-param, multi-param, nested-paren
(`SummingMergeTree((a, b))`), no-param, blank/null input, engine name
mismatch, `GraphiteMergeTree` quote preservation, and
`AggregatingMergeTree`
no-param.
- Docker IT: 5 round-trip tests on a real ClickHouse container covering
`ReplacingMergeTree`, `SummingMergeTree` (nested parens),
`VersionedCollapsingMergeTree` (multi-param), `CollapsingMergeTree`
(load from existing table), and `MergeTree` (no params). Each test verifies
both the Gravitino property value and the `SHOW CREATE TABLE` output.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
