jiangxt2 opened a new pull request, #11715: URL: https://github.com/apache/gravitino/pull/11715
### What changes were proposed in this pull request? This PR upgrades the Doris JDBC catalog's compatibility from Doris 1.2.x to Doris 3.0.x / 4.0.x. It includes six categories of fixes: 1. **Partition regex fix**: `DorisUtils.extractPartitionInfoFromSql()` regex now handles multi-line `SHOW CREATE TABLE` output and the space between `LIST`/`RANGE` and `(` in Doris 3.0+. Also extracts LIST partition assignments (values). 2. **INDEX syntax fix**: `appendIndexesSql()` and `addIndexDefinition()` no longer generate `INDEX PRIMARY (id)` syntax for PRIMARY_KEY/UNIQUE_KEY indexes. Added ANN index type mapping for Doris 3.1+ VECTOR indexes. 3. **datev2 fix**: `fromGravitino(DateType)` now outputs `datev2` instead of `date`, ensuring compatibility with Doris 4.0.x where `disable_datev1=true` is the default. 4. **DATETIME precision fix**: Added `calculateDatetimePrecision()` to parse `datetime(N)` format directly from `SHOW CREATE TABLE`, fixing precision loss when `COLUMN_SIZE`/`DECIMAL_DIGITS` are null for DATETIMEV2 columns. 5. **Type mapping extension**: `toGravitino` now recognizes binary/varbinary, array, map, struct, json, variant, ipv4/ipv6, largeint, bitmap, hll types. `fromGravitino` now supports ListType/MapType/StructType. Fixed `findCommaIndex` to track parenthesis depth for nested types like `decimal(10,2)`. 6. **Auto Increment support**: Removed the hard validation in `validateIncrementCol()` that blocked auto-increment columns. ### Why are the changes needed? Gravitino's Doris catalog declares support for Doris 1.2.x, which has been EOL for over two years. Users running Doris 3.0.x (LTS) or 4.0.x (stable) encounter multiple failures: partition operations broken, INDEX syntax errors, type mapping gaps, and auto-increment blocked. This PR brings compatibility in line with what users are running in production. Fix: #11590 ### Does this PR introduce _any_ user-facing change? - `fromGravitino(DateType)` now outputs `datev2` instead of `date` (Doris 1.2.x also supports `datev2`, so this is backward compatible). - New type mappings: binary/varbinary → BinaryType, array → ListType, map → MapType, struct → StructType, json/variant/ip → ExternalType. - Auto-increment columns are no longer blocked at the Gravitino layer (Doris will reject unsupported versions at execution time). ### How was this patch tested? - Unit tests: all existing + new tests pass (`./gradlew :catalogs:catalog-jdbc-doris:test -PskipITs`). - Integration tests: verified on both Doris 3.0.6.2 (30/31 pass, 1 version-limited) and Doris 4.0.6 (30/31 pass, 1 version-limited) with full CRUD, partition management, type mapping, index, auto-increment, and generated column scenarios. - Code formatting: `spotlessCheck` passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
