zhang-arvin opened a new pull request, #13393: URL: https://github.com/apache/gravitino/pull/13393
### What changes were proposed in this pull request? Preserve metadata for the legacy `annoy` and `usearch` ClickHouse data-skipping indexes on load. - Added `DATA_SKIPPING_ANNOY` and `DATA_SKIPPING_USEARCH` to `Index.IndexType`. - Mapped the ClickHouse `annoy` / `usearch` type names — bare and parameterized — to those two types, so they are no longer treated as unrecognized and skipped. - The load path now keeps the verbatim legacy type clause (`legacy_type`), plus the parsed `distance_function` and `dimensions`, in `Index.properties()`, so a legacy index stays distinguishable from the current `vector_similarity` HNSW index instead of being misrepresented as it. - DDL generation is deliberately not added. `CREATE TABLE`, `ALTER TABLE ADD INDEX` and table recreation now reject the legacy types with the index name and type in the message rather than translating them to `vector_similarity`. Migrating to `vector_similarity` remains an explicit user action. Scope matches the issue: metadata loading only, no legacy DDL generation. ### Why are the changes needed? The catalog currently warns and skips indexes whose type it does not recognize, which loses legacy index metadata when loading tables created by ClickHouse servers that still carried `annoy` / `usearch` (replaced by `vector_similarity` in v24.8, unsupported since v25.5). Fixes #13376 ### Does this PR introduce _any_ user-facing change? Yes. Tables with legacy `annoy` / `usearch` indexes now load with that index present in `Table.index()` as its own index type, with the legacy definition retained in `Index.properties()`. Attempting to create or alter such an index through Gravitino now fails with a clear error naming the index and the legacy type instead of a generic unsupported-index message. ### How was this patch tested? Added unit tests in `TestClickHouseTableOperationsIndexParsing` covering: - bare and parameterized `annoy` / `usearch` type mapping, and that the legacy types stay distinct from the other supported data-skipping types; - `parseLegacyVectorIndexProperties` retaining the verbatim type clause and the parsed distance function and dimensions, including the bare-type case, and leaving non-legacy types unaffected. `./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:test -PskipITs --tests "*IndexParsing*"` passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
