voonhous opened a new issue, #20210: URL: https://github.com/apache/hudi/issues/20210
### Task Description **What needs to be done:** Give hudi-trino one bidirectional type converter between Trino `Type` and `HoodieSchema`, owning both directions and the reject rules, with a round-trip test over every accepted type. After #20012, the type rules live in four places with no shared contract: | Concern | Where | Owner | |---|---|---| | Trino to Hudi (CREATE) | `HudiSchemaConverter.toHoodieSchema`, hand-rolled `instanceof` chain | hudi-trino | | Hudi to Trino (read, `register_table`) | `HudiUtil.toColumnHandle` via Trino's `NativeLogicalTypesAvroTypeBlockHandler.typeFor` | Trino library | | Nested Avro-name rules | `HudiTableValidation.validateNestedAvroNames` (walks Row/Array/Map again) | hudi-trino | | Hive-incompatible logical types | `HudiMetastoreTables.rejectLocalTimestamp` (walks the Avro schema again) | hudi-trino | Proposed shape (similar to `HoodieSparkSchemaConverters` in Spark, `HoodieSchemaConverter` in Flink and Trino Iceberg's `TypeConverter`): - `HudiTypeConverter.toHoodieSchema(Type)` and `toTrinoType(HoodieSchema)` in one class. - Reject rules (CHAR, TIME, UUID, TIMESTAMP WITH TIME ZONE, non-VARCHAR map keys, invalid Avro names, local-timestamp on register) decided in that class, so validation and metastore code call it instead of re-walking types. - Deliberate widenings (TINYINT/SMALLINT to INTEGER, VARCHAR(n) to VARCHAR, TIMESTAMP(p) rounding) listed in one table that the test also reads. - A table-driven round-trip test: for every accepted Trino type, `toTrinoType(toHoodieSchema(t))` equals `t` or its documented widening. Today `TestHudiSchemaConverter` only checks Hive registerability. **Why this task is needed:** The two directions are maintained by different code and nothing makes them agree. Each new type (Variant, nanosecond timestamps, row writing in later DDL/DML stages) has to be taught to up to four walkers separately, and a drift only shows up when another engine writes into a table Trino created. ### Task Type Code improvement/refactoring ### Related Issues **Parent feature issue:** #15527 **Related issues:** #20012 (review thread: https://github.com/apache/hudi/pull/20012#discussion_r4194904951) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
