zhang-arvin opened a new pull request, #13395: URL: https://github.com/apache/gravitino/pull/13395
### What changes were proposed in this pull request? Allow the ClickHouse catalog to create and alter text data-skipping indexes with DDL matching the connected server version. - `Index.IndexType` gains `DATA_SKIPPING_TEXT`, and the load path maps every text index keyword a supported release uses (`inverted`, `full_text`, `gin`, `text`) to it instead of skipping the index. - New `ClickHouseTextIndexDialect` resolves the connected server version and selects the grammar: `inverted(N)` (v23.2-v24.4), `full_text(N)` (v24.5 up to the gin rename), `gin(N)` (v25.5), the v2 key-value tokenizer grammar (v25.6-v25.8) and the v3 grammar (v25.9+). From v25.12 an explicit tokenizer is emitted because `default` is no longer accepted. - Both `CREATE TABLE` and `ALTER TABLE ADD INDEX` use the resolved grammar through the shared `buildDataSkippingIndexDdl` path. - `tokenizer` and `ngram_size` are read from the existing index property plumbing (`Index.properties()` / `TableChange.AddIndex.getProperties()`); `ngram_size` is validated against the common 2-8 range. - The text v3 default granularity (`100000000`) is applied for the v3 grammar. - An unclassifiable server version fails with a clear error, and no server-wide experimental setting is changed. ### Scope This PR covers version resolution, DDL dialect selection and CREATE + ALTER SQL generation, as the issue describes. The issue also asks for one focused Docker integration test per DDL boundary (24.4, 24.8.14, 25.5.1.2782, 25.6, 25.9, 26.2) and a full ClickHouse integration-suite run on the 24.8.14 baseline. Those integration tests are **not** included here — they need the multi-version Docker service wiring and are delivered separately. The unit tests below cover the version resolution, grammar boundaries, property validation and dialect selection that the integration tests would exercise. ### Why are the changes needed? The catalog currently rejects text data-skipping indexes in both table creation and `ALTER TABLE ADD INDEX`, and the server changed the index keyword and DDL grammar across releases, so a single hard-coded clause cannot work everywhere. Fixes #13375 ### Does this PR introduce _any_ user-facing change? Yes. Text data-skipping indexes can now be created through `CREATE TABLE` and `ALTER TABLE ADD INDEX` with `tokenizer` and `ngram_size` index properties, and the generated DDL adapts to the connected server version. Existing tables using any of the historical text index keywords now load with the index preserved. ### How was this patch tested? `TestClickHouseTextIndexDialect` covers version parsing (including the failure cases), grammar selection at every boundary (23.2, 24.4, 24.5, 24.8.14, 25.5.1, 25.6, 25.9, 25.12, 26.2), the legacy positional forms, the v2 and v3 tokenizer clauses, the mandatory-tokenizer rule from v25.12, the v3 default granularity, and `ngram_size` validation including the 2-8 range bounds. `./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:test -PskipITs` passes (106 tests). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
